-- Logs begin at Tue 2022-08-02 09:56:20 UTC, end at Tue 2022-08-02 11:13:59 UTC. -- Aug 02 10:34:09.538448 dv-cinder-850318-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 02 10:34:09.545826 dv-cinder-850318-6-iscsi devstack@keystone.service[92101]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 02 10:34:09.549394 dv-cinder-850318-6-iscsi devstack@keystone.service[92101]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 2 10:34:09 2022] *** Aug 02 10:34:09.549394 dv-cinder-850318-6-iscsi devstack@keystone.service[92101]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 02 10:34:09.549394 dv-cinder-850318-6-iscsi devstack@keystone.service[92101]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 02 10:34:09.549394 dv-cinder-850318-6-iscsi devstack@keystone.service[92101]: nodename: dv-cinder-850318-6-iscsi Aug 02 10:34:09.549394 dv-cinder-850318-6-iscsi devstack@keystone.service[92101]: machine: x86_64 Aug 02 10:34:09.549394 dv-cinder-850318-6-iscsi devstack@keystone.service[92101]: clock source: unix Aug 02 10:34:09.549394 dv-cinder-850318-6-iscsi devstack@keystone.service[92101]: pcre jit disabled Aug 02 10:34:09.549394 dv-cinder-850318-6-iscsi devstack@keystone.service[92101]: detected number of CPU cores: 4 Aug 02 10:34:09.549394 dv-cinder-850318-6-iscsi devstack@keystone.service[92101]: current working directory: / Aug 02 10:34:09.549394 dv-cinder-850318-6-iscsi devstack@keystone.service[92101]: detected binary path: /usr/bin/uwsgi-core Aug 02 10:34:09.549394 dv-cinder-850318-6-iscsi devstack@keystone.service[92101]: your processes number limit is 47680 Aug 02 10:34:09.549394 dv-cinder-850318-6-iscsi devstack@keystone.service[92101]: your memory page size is 4096 bytes Aug 02 10:34:09.549394 dv-cinder-850318-6-iscsi devstack@keystone.service[92101]: detected max file descriptor number: 2048 Aug 02 10:34:09.549394 dv-cinder-850318-6-iscsi devstack@keystone.service[92101]: lock engine: pthread robust mutexes Aug 02 10:34:09.549394 dv-cinder-850318-6-iscsi devstack@keystone.service[92101]: thunder lock: enabled Aug 02 10:34:09.549394 dv-cinder-850318-6-iscsi devstack@keystone.service[92101]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 02 10:34:09.549394 dv-cinder-850318-6-iscsi devstack@keystone.service[92101]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 02 10:34:09.590403 dv-cinder-850318-6-iscsi devstack@keystone.service[92101]: Python main interpreter initialized at 0x55e3700559c0 Aug 02 10:34:09.590403 dv-cinder-850318-6-iscsi devstack@keystone.service[92101]: python threads support enabled Aug 02 10:34:09.590403 dv-cinder-850318-6-iscsi devstack@keystone.service[92101]: your server socket listen backlog is limited to 100 connections Aug 02 10:34:09.590403 dv-cinder-850318-6-iscsi devstack@keystone.service[92101]: your mercy for graceful operations on workers is 90 seconds Aug 02 10:34:09.590794 dv-cinder-850318-6-iscsi devstack@keystone.service[92101]: mapped 403077 bytes (393 KB) for 2 cores Aug 02 10:34:09.590873 dv-cinder-850318-6-iscsi devstack@keystone.service[92101]: *** Operational MODE: preforking *** Aug 02 10:34:09.591140 dv-cinder-850318-6-iscsi devstack@keystone.service[92101]: *** uWSGI is running in multiple interpreter mode *** Aug 02 10:34:09.591140 dv-cinder-850318-6-iscsi devstack@keystone.service[92101]: spawned uWSGI master process (pid: 92101) Aug 02 10:34:09.591404 dv-cinder-850318-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 02 10:34:09.591995 dv-cinder-850318-6-iscsi devstack@keystone.service[92101]: spawned uWSGI worker 1 (pid: 92110, cores: 1) Aug 02 10:34:09.595900 dv-cinder-850318-6-iscsi devstack@keystone.service[92101]: spawned uWSGI worker 2 (pid: 92111, cores: 1) Aug 02 10:34:09.595900 dv-cinder-850318-6-iscsi devstack@keystone.service[92101]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 02 10:34:15.238584 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 02 10:34:15.238584 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: from cryptography.utils import int_from_bytes Aug 02 10:34:15.239501 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 02 10:34:15.239501 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: from cryptography.utils import int_from_bytes Aug 02 10:34:15.239782 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 02 10:34:15.239782 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: from cryptography.utils import int_from_bytes Aug 02 10:34:15.240266 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 02 10:34:15.240266 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: from cryptography.utils import int_from_bytes Aug 02 10:34:15.291477 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 02 10:34:15.291695 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 02 10:34:15.291849 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] command line args: [] {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 02 10:34:15.291952 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 02 10:34:15.292074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 02 10:34:15.292161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 02 10:34:15.292314 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] command line args: [] {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 02 10:34:15.292410 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ================================================================================ {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 02 10:34:15.292535 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 02 10:34:15.292615 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] admin_token = **** {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.292721 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ================================================================================ {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 02 10:34:15.292847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] config_dir = [] {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.292982 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] admin_token = **** {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.293079 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.293166 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] config_source = [] {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.293244 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] config_dir = [] {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.293316 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] debug = True {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.293399 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.293516 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.293769 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] config_source = [] {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.293769 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] debug = True {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.293887 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.293944 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.294105 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] insecure_debug = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.294193 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.294311 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.294396 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] insecure_debug = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.294562 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.294857 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.294857 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] list_limit = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.295215 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.295357 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] log_config_append = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.295598 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] list_limit = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.295801 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.295998 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] log_config_append = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.296150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] log_dir = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.296370 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.296632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] log_file = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.296912 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] log_dir = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.297196 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.297471 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] log_file = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.297471 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.298023 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.298317 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.298612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.298906 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.299208 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.299208 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.299751 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.299987 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.300953 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.300953 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.301378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.301560 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.301722 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.301888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.302100 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.302273 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.302437 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.302597 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.302747 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.302898 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.303074 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.303232 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.306404 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.306568 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.306705 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] notification_format = cadf {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.306845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] notification_format = cadf {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.306976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.307151 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.307340 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.115/identity {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.307464 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.115/identity {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.307589 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] publish_errors = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.307720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] publish_errors = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.307838 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.307959 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.308076 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.308191 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.308191 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.308457 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.308586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.308586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.308833 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.308985 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] standard_threads = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.309112 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.309112 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] standard_threads = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.309339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] strict_password_check = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.309464 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] strict_password_check = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.309464 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.309692 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.309814 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] use_eventlog = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.309944 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] use_eventlog = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.310068 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] use_journal = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.310185 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] use_journal = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.310302 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] use_json = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.310495 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] use_json = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.310615 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] use_stderr = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.310732 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] use_stderr = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.310849 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] use_syslog = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.310965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] use_syslog = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.311108 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] watch_log_file = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.311284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] watch_log_file = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 02 10:34:15.311412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.311538 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.311657 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.311774 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.311891 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.312017 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.312141 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.312257 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.312381 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.312511 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.312630 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.312769 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.312892 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.313016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.313134 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.313251 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.313374 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.313504 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.313504 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.313734 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.313850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.313968 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.314091 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.314221 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.314346 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.314464 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.314591 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.314709 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.314833 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.314964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.315098 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.315218 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.315335 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.315463 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.315591 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.315716 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.315868 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.315991 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.316109 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.316226 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.316415 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.316536 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.316662 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.316807 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.316926 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.317538 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.317686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.317805 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.317921 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.318037 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.318153 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.318276 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.318401 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.318531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.318650 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.318767 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.318884 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.319001 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.319171 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.319292 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.319415 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.319540 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.319641 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.319744 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.319850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.319955 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.320059 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.320158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.320280 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.320389 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.320495 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.320594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.320696 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.320805 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.320941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.321071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.321202 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.321352 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.321475 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.321594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.321711 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.321711 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.321930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.322048 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.322048 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.322256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.322379 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.322498 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.322612 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.322612 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.322845 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.322964 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.322964 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.323201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.323318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.323318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.323557 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.323674 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.323674 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.323900 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.324018 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.324143 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.324262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.324368 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.324473 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.324473 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.324703 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.324830 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.324830 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.325060 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.325177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.325177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.325396 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.325522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.325522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.325751 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.325876 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.325876 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.326117 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.326249 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.326249 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.326485 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.326605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.326605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.326825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.326954 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.327134 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.327262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.327382 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.327500 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.327619 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.327746 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.327865 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.327984 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.328101 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.328208 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.328324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.328451 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] cache.enabled = True {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.328582 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.328704 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.328704 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.329004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.329122 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.329266 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.329390 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.329516 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.329641 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.329769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.329888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.330018 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.330150 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.330271 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.330390 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.330508 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.330625 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.330743 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.330870 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.330991 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.331135 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.331256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.331425 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.331548 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.331688 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.331813 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.331937 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.332054 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] cache.enabled = True {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.332156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.332280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.332408 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.332527 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.332645 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.332762 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.332887 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.333007 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.333132 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.333252 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.333369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.333742 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.333864 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.333983 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.334109 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.334229 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.334345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.334462 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.334584 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.334723 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.334852 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.334982 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.335116 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.335236 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.335362 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.335481 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.335625 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.335747 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.335866 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] auth.external = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.335984 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.336102 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] auth.mapped = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.336220 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.336346 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.336468 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.336593 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.336713 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.336831 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] auth.password = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.336949 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.337088 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] auth.token = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.337212 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.337331 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.337449 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.337566 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] catalog.caching = True {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.337684 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.337809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.337930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.338047 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.338216 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.338349 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.338474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.338593 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.338710 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.338837 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.338957 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.339156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] credential.caching = True {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.339288 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.339409 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] credential.driver = sql {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.339409 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.339659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] auth.external = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.339790 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.339945 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] auth.mapped = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.340064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.340183 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.340303 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.340430 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.340557 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.340697 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] auth.password = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.340821 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.340940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] auth.token = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.341058 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.341185 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.341320 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.341450 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] catalog.caching = True {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.341611 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.341730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.341848 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.341976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.342104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.342242 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.342373 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.342492 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.342610 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.342738 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.342858 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.342993 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] credential.caching = True {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.343153 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] federation.caching = True {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.343274 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] credential.driver = sql {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.343406 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.343602 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.343713 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] federation.driver = sql {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.343812 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.343911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.344009 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.344116 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.344236 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.344355 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.344475 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.344592 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.344710 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.344836 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.344957 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.345082 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.345201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.345319 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.345438 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.345564 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.345684 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.345802 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.345919 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.346038 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] identity.caching = True {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.346155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.346282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.346402 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.346521 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.346647 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.346767 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.346885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] federation.caching = True {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.347011 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.347172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.347303 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] identity.driver = sql {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.347422 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] federation.driver = sql {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.347542 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.347660 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.347790 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.347910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.348028 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.348147 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.348249 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.348353 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.348494 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.348494 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.348708 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.348808 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.348924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.349043 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.349170 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.349290 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.349408 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.349526 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.349644 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.349762 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.349888 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] identity.caching = True {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.350008 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.350126 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.350259 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.350385 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.350488 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.350613 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.350731 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.350881 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.351021 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.351158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] identity.driver = sql {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.351280 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.351409 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.351529 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.351690 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.351810 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.351928 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.352046 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.352174 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.352294 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.352412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.352535 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.352655 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.352774 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.352936 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.353059 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.353177 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.353295 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.353413 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.353531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.353658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.353778 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.353899 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.354018 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.354136 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.354263 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.354383 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.354502 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.354502 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.password = **** {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.354722 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.354722 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.354994 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.355173 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.355275 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.355374 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.355472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.355570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.357860 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.358117 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.358207 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.358284 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.358441 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.358521 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.358702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.358788 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.358864 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.358939 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.359013 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.359116 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.359207 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.359286 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.359360 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.359435 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.359509 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.359509 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.359509 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.359713 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.359787 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.user = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.359861 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.359934 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.360008 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.360089 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.360166 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.360240 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.360314 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.password = **** {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.360388 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.360461 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.360547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.360624 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.360703 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.360779 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.360904 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.360984 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.361067 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.361144 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.361218 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.361292 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.361365 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.361446 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.361530 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.361606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.361748 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.361828 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.361903 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.361977 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.362060 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.362137 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.362212 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.362286 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.362360 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.362433 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.362515 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.363272 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.363360 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.363425 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.user = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.363547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.363628 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.363706 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.363771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.363831 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.363891 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.363950 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.364010 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.364077 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.364155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.364226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.364286 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.364345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.364404 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.364472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.364613 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.364697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] policy.driver = sql {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.364772 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.364893 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.364973 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.365113 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.365195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.365270 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.365345 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.365418 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] receipt.caching = True {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.365492 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.365575 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.365652 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.365726 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.365800 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.365873 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.365873 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.366083 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.366163 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.366238 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.366312 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] resource.caching = True {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.366386 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.366460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.366545 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.366623 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] resource.driver = sql {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.366698 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.366772 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.366845 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.366919 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.367002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.367094 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.367171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.367245 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] revoke.caching = True {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.367319 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.367405 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.367484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.367558 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.367632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.367706 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] role.cache_time = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.367779 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] policy.driver = sql {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.367861 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] role.caching = True {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.367938 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.368012 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] role.driver = sql {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.368086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.368160 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] role.list_limit = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.368233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.368233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] receipt.caching = True {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.368377 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.368451 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.368525 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.368600 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.368674 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.368756 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.368832 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.368906 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.368980 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.369054 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.369127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.369224 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] resource.caching = True {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.369307 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.369383 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.369457 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.369530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] resource.driver = sql {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.369604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.369686 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.369761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.369836 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.369930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.370006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.370128 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.370216 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] revoke.caching = True {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.370293 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.370367 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.370367 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.370500 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.370574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] role.cache_time = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.370656 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.370825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] role.caching = True {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.370912 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.370990 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] role.driver = sql {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.371079 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.371156 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] role.list_limit = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.371247 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.371326 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.371418 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.371514 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.371591 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.371666 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.371749 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.371813 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.371935 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.372021 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.372098 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.372171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.372253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.372253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.372390 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.372463 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.372537 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.372618 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.372695 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.372769 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.372842 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.372915 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.372988 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.373069 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.373146 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.373219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.373293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.373366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] token.cache_time = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.373366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] token.caching = True {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.373507 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.373583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.373658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.373731 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] token.provider = fernet {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.373804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.373878 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.373959 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.374035 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.374109 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.374182 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.374255 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.374328 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.374409 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.374492 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.374567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.374641 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.374714 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.374788 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] trust.driver = sql {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.374870 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.374945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.375019 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.375134 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.375198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.375258 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.375387 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.375501 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.375644 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.375781 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.375906 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.376050 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.376211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.376333 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.376505 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.376707 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.376954 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.377105 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] database.connection = **** {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.377214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] token.cache_time = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.377300 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.377396 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] token.caching = True {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.377587 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.377777 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.377984 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.378082 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] token.provider = fernet {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.378156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.378853 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.378996 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.379111 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.379185 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.379253 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.379393 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.379481 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.379565 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.379648 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.379730 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.379811 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.379899 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.380002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] trust.driver = sql {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.380081 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.380162 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.380252 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.380341 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.380432 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.380530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.380618 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.380709 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.380809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.380898 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.380988 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.381072 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.381157 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.381243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.381327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.381327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92111) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 02 10:34:15.381483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.381483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] database.connection = **** {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.381483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.381483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.381849 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.381849 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.382026 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.382265 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.382502 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.382741 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.382988 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.383283 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.383531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.383768 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.384017 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.384256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.384546 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.384781 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.385024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.385263 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 02 10:34:15.385435 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=92110) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 02 10:34:15.421731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=92110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.421974 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=92110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.422061 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=92111) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.422149 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=92110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.422236 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=92111) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.422469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=92111) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.432133 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=92111) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.432406 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=92110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.464861 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=92110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.470858 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=92110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.471029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=92111) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.471029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=92111) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.526205 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=92110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.527901 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=92111) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.579179 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=92110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.579416 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=92110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.586552 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=92111) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.586800 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=92111) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.648234 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.648461 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92111) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.648461 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=92111) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.648610 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=92110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.648676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=92111) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.648738 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=92110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.648876 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=92110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.648964 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=92111) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.649100 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=92111) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.649216 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=92110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.649299 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=92110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.649376 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=92111) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.674115 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.674329 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92111) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.674429 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=92110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.674514 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=92111) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.674616 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=92110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.674706 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=92111) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.674790 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=92110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.674950 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=92111) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.675039 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=92110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.675312 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=92111) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.675901 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=92110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.675971 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=92111) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.676066 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.676194 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92111) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.687932 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=92110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.688181 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=92111) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.701504 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.701725 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=92110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.701988 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=92111) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.702192 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=92110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.702192 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=92110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.702418 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=92111) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.702493 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=92110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.702607 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=92111) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.702732 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=92110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.702865 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=92111) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.703009 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92110) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.703510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=92111) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.703510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=92111) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.703510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=92111) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 02 10:34:15.720845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.721064 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.721187 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 02 10:34:15.721290 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 02 10:34:15.721419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.721516 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.721636 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 02 10:34:15.721748 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 02 10:34:15.721904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.722539 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.722744 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.722845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.723017 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.723218 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.723377 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.723508 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.733172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.733453 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.733606 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.733723 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.733901 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.733988 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.734178 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.734469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.741904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=92110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.742293 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=92111) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.746964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=92110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.747265 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.747430 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=92111) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.747531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.747762 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.747854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.748062 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.748172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.748330 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.748426 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.748612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.748780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.748879 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.748975 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.749106 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.749205 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.749385 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.749491 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.749652 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.749752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.749926 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.750221 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.774739 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.775261 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.776359 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=92110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.776596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.776893 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=92111) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.777179 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.781266 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=92110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.781545 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.781824 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.781914 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=92111) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.782197 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.782484 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.788791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=92110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.789084 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.790079 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=92111) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.790375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.793361 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=92110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.793643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.793938 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.794223 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.794491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.794767 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.794883 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=92111) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.795062 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.795151 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.795344 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.795428 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.795644 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.795935 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.796233 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.796507 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.796789 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.811426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.811704 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.813937 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.814226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.816237 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=92110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.818883 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=92111) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.819529 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.819824 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.822291 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.822591 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.824365 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=92110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.827230 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=92111) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.828539 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=92110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.830486 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=92111) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.832407 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.832710 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.833020 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.833289 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.833840 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.834124 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.834880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.835215 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.835521 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.835855 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.836152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.836443 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.850772 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=92110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.851088 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.851363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.851624 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.854261 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=92111) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.854583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.854855 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.855156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.858606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.860196 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.860466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.862482 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.863936 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=92110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.864214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.864513 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.864639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.864801 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.864930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.865086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.867814 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=92111) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.868099 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.868392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.868677 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.869017 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.876642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=92110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.876918 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.877193 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.877470 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.877747 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.878019 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.878975 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.882211 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=92111) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.882533 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.882819 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.883130 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.883420 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.883745 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.884070 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.894798 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=92110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.897807 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=92110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.900831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.901560 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=92111) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.903690 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.904950 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=92111) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.905052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=92110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.905323 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.905603 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.908060 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.910968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.912218 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.912711 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=92111) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.913001 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.913287 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.913817 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=92110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.916787 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.917071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.917355 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.917631 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.920235 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.921860 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=92111) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.924937 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.925849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.926077 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.926355 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.926453 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=92110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.926691 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.926969 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.933930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=92110) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.934207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.934483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.934767 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.935040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.935142 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=92111) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.935349 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.935438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.935621 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.935713 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.935887 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.936268 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.936547 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.936830 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.937091 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.937362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.937622 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=92110) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.943903 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=92111) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 02 10:34:15.944263 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.944557 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.944856 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.945190 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.945487 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.945835 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.946120 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.946563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.946866 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.947196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.947507 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.947790 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.948057 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=92111) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 02 10:34:15.973084 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e3700559c0 pid: 92110 (default app) Aug 02 10:34:15.976678 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27e8dd-7fb9-4239-b936-1b963090df97 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:15.980273 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27e8dd-7fb9-4239-b936-1b963090df97 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:15.980273 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b27e8dd-7fb9-4239-b936-1b963090df97 None None] PATH_INFO: `/v3` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:15.981799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1/1] 10.222.0.115 () {50 vars in 796 bytes} [Tue Aug 2 10:34:15 2022] GET /identity/v3/ => generated 256 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:15.983606 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e3700559c0 pid: 92111 (default app) Aug 02 10:34:25.069428 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2820949-40cf-4b33-97f5-e4ae96f38334 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:25.070011 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2820949-40cf-4b33-97f5-e4ae96f38334 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:25.070011 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2820949-40cf-4b33-97f5-e4ae96f38334 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:25.071642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 2/2] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:34:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:34:25.079529 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e178b-73f5-420c-9f88-f936b4811e15 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:25.080061 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e178b-73f5-420c-9f88-f936b4811e15 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:25.080061 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e178b-73f5-420c-9f88-f936b4811e15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:25.151906 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: INFO dbcounter [None req-a27e178b-73f5-420c-9f88-f936b4811e15 None None] Registered counter for database keystone Aug 02 10:34:25.173559 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG oslo_db.sqlalchemy.engines [None req-a27e178b-73f5-420c-9f88-f936b4811e15 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=92111) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 02 10:34:25.180507 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writer thread running {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 02 10:34:25.366417 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG passlib.handlers.bcrypt [None req-a27e178b-73f5-420c-9f88-f936b4811e15 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=92111) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 02 10:34:25.366751 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG passlib.handlers.bcrypt [None req-a27e178b-73f5-420c-9f88-f936b4811e15 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=92111) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 02 10:34:25.422007 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-a27e178b-73f5-420c-9f88-f936b4811e15 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:25.423678 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-a27e178b-73f5-420c-9f88-f936b4811e15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:25.564834 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1/3] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:34:25 2022] POST /identity/v3/auth/tokens => generated 915 bytes in 489 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 02 10:34:25.689813 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2820949-40cf-4b33-97f5-e4ae96f38334 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:25.759486 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: INFO dbcounter [None req-f2820949-40cf-4b33-97f5-e4ae96f38334 None None] Registered counter for database keystone Aug 02 10:34:25.779610 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG oslo_db.sqlalchemy.engines [None req-f2820949-40cf-4b33-97f5-e4ae96f38334 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=92110) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 02 10:34:25.786425 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writer thread running {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 02 10:34:25.904444 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2820949-40cf-4b33-97f5-e4ae96f38334 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:25.908692 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b5273-4ded-424a-9a33-e0d639dd288e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:25.909319 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b5273-4ded-424a-9a33-e0d639dd288e admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:25.909632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b5273-4ded-424a-9a33-e0d639dd288e admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:25.909907 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b5273-4ded-424a-9a33-e0d639dd288e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:25.918738 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b5273-4ded-424a-9a33-e0d639dd288e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:25.974429 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e44b5273-4ded-424a-9a33-e0d639dd288e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 02 10:34:25.975221 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b5273-4ded-424a-9a33-e0d639dd288e admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:25.985895 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 3/4] 10.222.0.115 () {58 vars in 1329 bytes} [Tue Aug 2 10:34:25 2022] GET /identity/v3/auth/tokens => generated 915 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 02 10:34:25.993747 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e178b-73f5-420c-9f88-f936b4811e15 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:26.009713 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e178b-73f5-420c-9f88-f936b4811e15 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:26.014225 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec25a8-2409-4e97-b74f-5ca39a50926c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:26.015373 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec25a8-2409-4e97-b74f-5ca39a50926c admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:26.015656 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec25a8-2409-4e97-b74f-5ca39a50926c admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:26.015928 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec25a8-2409-4e97-b74f-5ca39a50926c admin admin] PATH_INFO: `/v3/projects/b86f954a6cac457bad0b226e93315c59` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:26.017551 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ec25a8-2409-4e97-b74f-5ca39a50926c admin admin] RBAC: Authorizing `identity:get_project(project_id=b86f954a6cac457bad0b226e93315c59)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:26.069028 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e2ec25a8-2409-4e97-b74f-5ca39a50926c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 02 10:34:26.069359 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ec25a8-2409-4e97-b74f-5ca39a50926c admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:26.074145 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 2/5] 10.222.0.115 () {56 vars in 1212 bytes} [Tue Aug 2 10:34:25 2022] GET /identity/v3/projects/b86f954a6cac457bad0b226e93315c59 => generated 345 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:34:28.096170 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf671f9-c353-4f0a-a747-a39406131f93 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:28.096724 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf671f9-c353-4f0a-a747-a39406131f93 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:28.096823 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf671f9-c353-4f0a-a747-a39406131f93 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:28.101522 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 4/6] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:34:28 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:34:28.106872 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcdb3f-c4e1-4980-addf-9e594625787f None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:28.107311 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcdb3f-c4e1-4980-addf-9e594625787f None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:28.107464 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbcdb3f-c4e1-4980-addf-9e594625787f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:28.128499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-0bbcdb3f-c4e1-4980-addf-9e594625787f None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:28.130086 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-0bbcdb3f-c4e1-4980-addf-9e594625787f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:28.138277 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 3/7] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:34:28 2022] POST /identity/v3/auth/tokens => generated 915 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Aug 02 10:34:28.161464 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf671f9-c353-4f0a-a747-a39406131f93 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:28.173402 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf671f9-c353-4f0a-a747-a39406131f93 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:28.178697 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b81d8b-181e-47e7-bbcd-49ea336a7543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:28.179330 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b81d8b-181e-47e7-bbcd-49ea336a7543 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:28.179613 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b81d8b-181e-47e7-bbcd-49ea336a7543 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:28.179886 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b81d8b-181e-47e7-bbcd-49ea336a7543 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:28.187643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b81d8b-181e-47e7-bbcd-49ea336a7543 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:28.189462 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b81d8b-181e-47e7-bbcd-49ea336a7543 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:28.200633 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 5/8] 10.222.0.115 () {58 vars in 1329 bytes} [Tue Aug 2 10:34:28 2022] GET /identity/v3/auth/tokens => generated 915 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:34:28.207172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbcdb3f-c4e1-4980-addf-9e594625787f None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:28.216520 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbcdb3f-c4e1-4980-addf-9e594625787f None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:28.221010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c05ee1a-ef5b-405e-b132-8c44c22f32ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:28.221634 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05ee1a-ef5b-405e-b132-8c44c22f32ca admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:28.221942 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05ee1a-ef5b-405e-b132-8c44c22f32ca admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:28.222218 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c05ee1a-ef5b-405e-b132-8c44c22f32ca admin admin] PATH_INFO: `/v3/users/4fb0c392df554f5dbd68f6861be954a1` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:28.223615 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05ee1a-ef5b-405e-b132-8c44c22f32ca admin admin] RBAC: Authorizing `identity:get_user(user_id=4fb0c392df554f5dbd68f6861be954a1)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:28.225450 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c05ee1a-ef5b-405e-b132-8c44c22f32ca admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:28.227024 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 4/9] 10.222.0.115 () {56 vars in 1203 bytes} [Tue Aug 2 10:34:28 2022] GET /identity/v3/users/4fb0c392df554f5dbd68f6861be954a1 => generated 248 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:30.321931 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee39fd-e4b7-4564-b2a7-79b6fc92e413 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:30.322502 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee39fd-e4b7-4564-b2a7-79b6fc92e413 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:30.322502 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee39fd-e4b7-4564-b2a7-79b6fc92e413 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:30.323842 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 6/10] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:34:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:34:30.329095 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37abd18-d3ed-408c-a368-1fc2c000a662 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:30.329504 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37abd18-d3ed-408c-a368-1fc2c000a662 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:30.329660 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37abd18-d3ed-408c-a368-1fc2c000a662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:30.347978 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565e1ba-187c-4b3e-a657-236ec5138f8a None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:30.348358 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565e1ba-187c-4b3e-a657-236ec5138f8a None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:30.348513 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6565e1ba-187c-4b3e-a657-236ec5138f8a None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:30.349854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 7/11] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:34:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:34:30.350357 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-b37abd18-d3ed-408c-a368-1fc2c000a662 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:30.351691 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-b37abd18-d3ed-408c-a368-1fc2c000a662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:30.355007 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd1c33-dea7-44d6-a51d-bfef71551033 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:30.355356 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd1c33-dea7-44d6-a51d-bfef71551033 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:30.355632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd1c33-dea7-44d6-a51d-bfef71551033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:30.399182 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG passlib.handlers.bcrypt [None req-f7fd1c33-dea7-44d6-a51d-bfef71551033 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=92110) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 02 10:34:30.399617 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG passlib.handlers.bcrypt [None req-f7fd1c33-dea7-44d6-a51d-bfef71551033 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=92110) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 02 10:34:30.434369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 5/12] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:34:30 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:34:30.445440 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-f7fd1c33-dea7-44d6-a51d-bfef71551033 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:30.447094 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-f7fd1c33-dea7-44d6-a51d-bfef71551033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:30.458390 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37abd18-d3ed-408c-a368-1fc2c000a662 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:30.482411 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:30.551594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 8/13] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:34:30 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:34:30.563379 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37abd18-d3ed-408c-a368-1fc2c000a662 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:30.573222 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd1c33-dea7-44d6-a51d-bfef71551033 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:30.597574 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a36e1-6cb0-403e-bd06-c94fbf0f07b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:30.598222 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a36e1-6cb0-403e-bd06-c94fbf0f07b2 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:30.598510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a36e1-6cb0-403e-bd06-c94fbf0f07b2 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:30.598783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a36e1-6cb0-403e-bd06-c94fbf0f07b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:30.610703 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a36e1-6cb0-403e-bd06-c94fbf0f07b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:34:30.619319 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-604a36e1-6cb0-403e-bd06-c94fbf0f07b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:34:30.619319 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:34:30.619319 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:34:30.619319 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:34:30.619319 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:34:30.619319 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:34:30.619319 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:34:30.619319 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:34:30.619319 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:34:30.619319 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:34:30.619319 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:34:30.619319 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:34:30.619319 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:34:30.619319 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:34:30.619319 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:34:30.619319 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:34:30.620270 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:34:30.620270 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:34:30.620270 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:34:30.620270 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:34:30.620270 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:34:30.620270 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:34:30.620270 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:34:30.620270 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:34:30.620270 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:34:30.620270 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:34:30.620270 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:34:30.620270 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:34:30.620270 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:34:30.620270 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:34:30.620270 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:34:30.620270 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:34:30.621325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:34:30.621325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:34:30.621325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:34:30.621325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:34:30.621325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:34:30.621325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:34:30.621325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:34:30.621325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:34:30.621325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:34:30.621325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:34:30.621325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:34:30.621325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:34:30.621325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 6/14] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:34:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:34:30.626281 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a36e1-6cb0-403e-bd06-c94fbf0f07b2 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:30.679577 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd1c33-dea7-44d6-a51d-bfef71551033 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:30.681818 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:30.712112 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:30.717959 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3870953c-1d95-4eae-a372-a3dd7d362b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:30.718834 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870953c-1d95-4eae-a372-a3dd7d362b18 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:30.719150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870953c-1d95-4eae-a372-a3dd7d362b18 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:30.719431 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3870953c-1d95-4eae-a372-a3dd7d362b18 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:30.730733 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a36e1-6cb0-403e-bd06-c94fbf0f07b2 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:30.765783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576a4bb-76a8-46d5-a4dc-d014267c968b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:30.766413 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576a4bb-76a8-46d5-a4dc-d014267c968b None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:30.766694 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576a4bb-76a8-46d5-a4dc-d014267c968b None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:30.766968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f576a4bb-76a8-46d5-a4dc-d014267c968b None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:30.767729 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f576a4bb-76a8-46d5-a4dc-d014267c968b None admin] RBAC: Adding query filter params (name=admin) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:34:30.768039 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f576a4bb-76a8-46d5-a4dc-d014267c968b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:30.770013 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f576a4bb-76a8-46d5-a4dc-d014267c968b None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:30.782637 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 7/15] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:34:30 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:34:30.790070 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576a4bb-76a8-46d5-a4dc-d014267c968b None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:30.791333 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3870953c-1d95-4eae-a372-a3dd7d362b18 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:30.792982 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3870953c-1d95-4eae-a372-a3dd7d362b18 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:30.900847 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f576a4bb-76a8-46d5-a4dc-d014267c968b None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:30.903111 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 9/16] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:34:30 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:34:30.908806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3870953c-1d95-4eae-a372-a3dd7d362b18 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:30.912697 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:30.937222 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7219fe-6f4d-4c7e-bbd8-6f8cea0863c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:30.937857 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7219fe-6f4d-4c7e-bbd8-6f8cea0863c0 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:30.938137 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7219fe-6f4d-4c7e-bbd8-6f8cea0863c0 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:30.938409 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7219fe-6f4d-4c7e-bbd8-6f8cea0863c0 None admin] PATH_INFO: `/v3/users/4fb0c392df554f5dbd68f6861be954a1` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:30.939611 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7219fe-6f4d-4c7e-bbd8-6f8cea0863c0 None admin] RBAC: Authorizing `identity:get_user(user_id=4fb0c392df554f5dbd68f6861be954a1)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:30.941604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7219fe-6f4d-4c7e-bbd8-6f8cea0863c0 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:30.943618 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 8/17] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:34:30 2022] GET /identity/v3/users/4fb0c392df554f5dbd68f6861be954a1 => generated 248 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:34:30.948643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7219fe-6f4d-4c7e-bbd8-6f8cea0863c0 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:30.964291 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:31.012186 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3870953c-1d95-4eae-a372-a3dd7d362b18 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:31.051980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb7d4e0-4d81-476b-a840-cc004dea4e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:31.052867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7d4e0-4d81-476b-a840-cc004dea4e2f None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:31.053145 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7d4e0-4d81-476b-a840-cc004dea4e2f None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:31.053414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb7d4e0-4d81-476b-a840-cc004dea4e2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:31.075117 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb7d4e0-4d81-476b-a840-cc004dea4e2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:31.076615 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb7d4e0-4d81-476b-a840-cc004dea4e2f None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:31.097028 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-bfb7d4e0-4d81-476b-a840-cc004dea4e2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:34:31.097028 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:34:31.097028 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:34:31.097028 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:34:31.097028 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:34:31.097028 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:34:31.097028 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:34:31.097028 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:34:31.097028 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:34:31.097028 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:34:31.097028 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:34:31.097028 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:34:31.097028 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:34:31.097028 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:34:31.097028 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:34:31.098427 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:34:31.098427 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:34:31.098427 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:34:31.098427 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:34:31.098427 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:34:31.098427 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:34:31.098427 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:34:31.098427 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:34:31.098427 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:34:31.098427 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:34:31.098427 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:34:31.098427 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:34:31.098427 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:34:31.098427 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:34:31.098427 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:34:31.098427 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:34:31.098427 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:34:31.105192 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:34:31.105192 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:34:31.105192 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:34:31.105192 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:34:31.105192 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 10/18] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:34:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:34:31.105465 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb7d4e0-4d81-476b-a840-cc004dea4e2f None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:31.158709 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7219fe-6f4d-4c7e-bbd8-6f8cea0863c0 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:31.162233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:31.193628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316e797-efb4-4c4b-9904-5fe5f2ddd2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:31.194269 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316e797-efb4-4c4b-9904-5fe5f2ddd2ae None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:31.194825 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316e797-efb4-4c4b-9904-5fe5f2ddd2ae None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:31.195145 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316e797-efb4-4c4b-9904-5fe5f2ddd2ae None admin] PATH_INFO: `/v3/domains/default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:31.196305 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9316e797-efb4-4c4b-9904-5fe5f2ddd2ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:31.197763 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9316e797-efb4-4c4b-9904-5fe5f2ddd2ae None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:31.199389 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 9/19] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:34:30 2022] GET /identity/v3/domains/default => generated 200 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:34:31.205299 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316e797-efb4-4c4b-9904-5fe5f2ddd2ae None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:31.207901 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb7d4e0-4d81-476b-a840-cc004dea4e2f None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:31.242789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e0c26-24f2-4317-b7c9-fc2d97636503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:31.243458 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e0c26-24f2-4317-b7c9-fc2d97636503 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:31.243742 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e0c26-24f2-4317-b7c9-fc2d97636503 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:31.244023 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e0c26-24f2-4317-b7c9-fc2d97636503 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:31.244754 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62e0c26-24f2-4317-b7c9-fc2d97636503 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:34:31.245058 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62e0c26-24f2-4317-b7c9-fc2d97636503 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:31.246458 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62e0c26-24f2-4317-b7c9-fc2d97636503 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:31.265006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 11/20] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:34:31 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:31.294654 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:31.315003 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316e797-efb4-4c4b-9904-5fe5f2ddd2ae None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:31.353919 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898720b2-e5c0-4b6a-80fe-9956daa3701a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:31.354573 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898720b2-e5c0-4b6a-80fe-9956daa3701a None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:31.354819 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898720b2-e5c0-4b6a-80fe-9956daa3701a None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:31.355125 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898720b2-e5c0-4b6a-80fe-9956daa3701a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:31.355931 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898720b2-e5c0-4b6a-80fe-9956daa3701a None admin] RBAC: Adding query filter params (role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac, scope.domain.id=default, user.id=4fb0c392df554f5dbd68f6861be954a1) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:34:31.356243 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898720b2-e5c0-4b6a-80fe-9956daa3701a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:31.357704 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898720b2-e5c0-4b6a-80fe-9956daa3701a None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:31.365948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 10/21] 10.222.0.115 () {56 vars in 1327 bytes} [Tue Aug 2 10:34:31 2022] GET /identity/v3/role_assignments?user.id=4fb0c392df554f5dbd68f6861be954a1&scope.domain.id=default&role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 234 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:34:33.808011 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8bd47-1373-4403-a690-352a7c7528dc None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:33.808523 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8bd47-1373-4403-a690-352a7c7528dc None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:33.808595 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8bd47-1373-4403-a690-352a7c7528dc None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:33.810288 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 12/22] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:34:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:34:33.815261 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b66edf-b283-4e95-b7fa-c6039a4a2db8 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:33.816720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b66edf-b283-4e95-b7fa-c6039a4a2db8 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:33.817236 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b66edf-b283-4e95-b7fa-c6039a4a2db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:33.841139 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-e3b66edf-b283-4e95-b7fa-c6039a4a2db8 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:33.842749 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-e3b66edf-b283-4e95-b7fa-c6039a4a2db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:33.923338 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 11/23] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:34:33 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:34:33.957799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e8bd47-1373-4403-a690-352a7c7528dc None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:34.085009 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e8bd47-1373-4403-a690-352a7c7528dc None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:34.107409 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:34.107827 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a4d05-0ebe-440a-b361-d7a9adee323b None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:34.108374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a4d05-0ebe-440a-b361-d7a9adee323b None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:34.108871 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107a4d05-0ebe-440a-b361-d7a9adee323b None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:34.115263 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 12/24] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:34:34 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:34:34.116899 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e2ba1-49c8-4079-9b1b-cbc43a443264 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:34.117391 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e2ba1-49c8-4079-9b1b-cbc43a443264 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:34.117907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e2ba1-49c8-4079-9b1b-cbc43a443264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:34.134457 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56917133-27e1-42b4-bfff-1769d0c42370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:34.135163 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56917133-27e1-42b4-bfff-1769d0c42370 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:34.135445 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56917133-27e1-42b4-bfff-1769d0c42370 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:34.135799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56917133-27e1-42b4-bfff-1769d0c42370 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:34.140247 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-0f3e2ba1-49c8-4079-9b1b-cbc43a443264 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:34.141876 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-0f3e2ba1-49c8-4079-9b1b-cbc43a443264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:34.156541 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56917133-27e1-42b4-bfff-1769d0c42370 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:34:34.164085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-56917133-27e1-42b4-bfff-1769d0c42370 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:34:34.164085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:34:34.164085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:34:34.164085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:34:34.164085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:34:34.164085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:34:34.164085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:34:34.164085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:34:34.164085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:34:34.164085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:34:34.164085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:34:34.164085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:34:34.164085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:34:34.164085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:34:34.164085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:34:34.164085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:34:34.164966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:34:34.164966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:34:34.164966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:34:34.164966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:34:34.164966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:34:34.164966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:34:34.164966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:34:34.164966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:34:34.164966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:34:34.164966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:34:34.164966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:34:34.164966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:34:34.164966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:34:34.164966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:34:34.164966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:34:34.164966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:34:34.165980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:34:34.165980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:34:34.165980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:34:34.165980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:34:34.165980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:34:34.165980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:34:34.165980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:34:34.165980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:34:34.165980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:34:34.165980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:34:34.165980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:34:34.165980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:34:34.165980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 13/25] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:34:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:34:34.171085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56917133-27e1-42b4-bfff-1769d0c42370 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:34.260699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 13/26] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:34:34 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:34:34.301107 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e2ba1-49c8-4079-9b1b-cbc43a443264 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:34.304652 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:34.399213 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56917133-27e1-42b4-bfff-1769d0c42370 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:34.502136 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd404829-6278-4d9f-9f66-5f9f7431fcba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:34.502837 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd404829-6278-4d9f-9f66-5f9f7431fcba None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:34.503392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd404829-6278-4d9f-9f66-5f9f7431fcba None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:34.503764 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd404829-6278-4d9f-9f66-5f9f7431fcba None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:34.504511 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd404829-6278-4d9f-9f66-5f9f7431fcba None admin] RBAC: Adding query filter params (name=admin) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:34:34.504821 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd404829-6278-4d9f-9f66-5f9f7431fcba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:34.506229 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd404829-6278-4d9f-9f66-5f9f7431fcba None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:34.520547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e2ba1-49c8-4079-9b1b-cbc43a443264 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:34.541220 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 14/27] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:34:34 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:34.547029 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd404829-6278-4d9f-9f66-5f9f7431fcba None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:34.612787 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74623306-7680-481b-aa04-8dad30f02fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:34.613448 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74623306-7680-481b-aa04-8dad30f02fe5 None admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:34.613734 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74623306-7680-481b-aa04-8dad30f02fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:34.614009 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74623306-7680-481b-aa04-8dad30f02fe5 None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:34.614764 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74623306-7680-481b-aa04-8dad30f02fe5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:34.616444 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74623306-7680-481b-aa04-8dad30f02fe5 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:34.718294 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:34.820696 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd404829-6278-4d9f-9f66-5f9f7431fcba None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:34.859221 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d77082-9288-46d6-86a5-39ffe40cdf1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:34.859873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d77082-9288-46d6-86a5-39ffe40cdf1d None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:34.860212 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d77082-9288-46d6-86a5-39ffe40cdf1d None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:34.860543 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d77082-9288-46d6-86a5-39ffe40cdf1d None admin] PATH_INFO: `/v3/users/4fb0c392df554f5dbd68f6861be954a1` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:34.862087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d77082-9288-46d6-86a5-39ffe40cdf1d None admin] RBAC: Authorizing `identity:get_user(user_id=4fb0c392df554f5dbd68f6861be954a1)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:34.863844 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d77082-9288-46d6-86a5-39ffe40cdf1d None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:34.865857 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 15/28] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:34:34 2022] GET /identity/v3/users/4fb0c392df554f5dbd68f6861be954a1 => generated 248 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:34:34.866939 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8bf0a-eb6b-4a8d-92d8-6291aa463ef1 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:34.867295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8bf0a-eb6b-4a8d-92d8-6291aa463ef1 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:34.867573 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b8bf0a-eb6b-4a8d-92d8-6291aa463ef1 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:34.868989 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 16/29] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:34:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:34:34.871021 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8bf0a-eb6b-4a8d-92d8-6291aa463ef1 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:34.922774 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 14/30] 10.222.0.115 () {60 vars in 1136 bytes} [Tue Aug 2 10:34:34 2022] POST /identity/v3/roles => generated 220 bytes in 622 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:34:34.924192 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee740f0b-a8fc-4d2d-b933-be90c0292dbd None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:34.924482 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee740f0b-a8fc-4d2d-b933-be90c0292dbd None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:34.924835 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee740f0b-a8fc-4d2d-b933-be90c0292dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:34.946038 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-ee740f0b-a8fc-4d2d-b933-be90c0292dbd None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:34.947935 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-ee740f0b-a8fc-4d2d-b933-be90c0292dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:34.999992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b8bf0a-eb6b-4a8d-92d8-6291aa463ef1 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:35.010045 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:35.030371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:35.035737 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 15/31] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:34:34 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:34:35.038010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d38598-2b8f-46b4-9107-e7aa92d2813c None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:35.038472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d38598-2b8f-46b4-9107-e7aa92d2813c None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:35.038911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d38598-2b8f-46b4-9107-e7aa92d2813c None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:35.041261 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 16/32] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:34:35 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:34:35.042190 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24012fdb-c027-4177-b30c-f093d2ef30e0 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:35.042474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24012fdb-c027-4177-b30c-f093d2ef30e0 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:35.042749 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24012fdb-c027-4177-b30c-f093d2ef30e0 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:35.044980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6bbb96-95d8-48b2-bb6c-da753fffb879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:35.045676 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6bbb96-95d8-48b2-bb6c-da753fffb879 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:35.045957 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6bbb96-95d8-48b2-bb6c-da753fffb879 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:35.046227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6bbb96-95d8-48b2-bb6c-da753fffb879 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:35.047443 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6bbb96-95d8-48b2-bb6c-da753fffb879 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:35.048917 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6bbb96-95d8-48b2-bb6c-da753fffb879 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:35.051807 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 17/33] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:34:34 2022] GET /identity/v3/domains/default => generated 200 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:35.053685 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 17/34] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:34:35 2022] GET /identity => generated 271 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:34:35.059108 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24012fdb-c027-4177-b30c-f093d2ef30e0 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:35.061065 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c0d33-f05b-4cf6-9055-660a97bb2f89 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:35.061065 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c0d33-f05b-4cf6-9055-660a97bb2f89 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:35.061065 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8c0d33-f05b-4cf6-9055-660a97bb2f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:35.086464 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-1d8c0d33-f05b-4cf6-9055-660a97bb2f89 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:35.088505 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-1d8c0d33-f05b-4cf6-9055-660a97bb2f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:35.185687 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 18/35] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:34:35 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:34:35.188222 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42000b20-f687-409d-ae34-1f3f08427224 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:35.188717 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42000b20-f687-409d-ae34-1f3f08427224 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:35.189187 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42000b20-f687-409d-ae34-1f3f08427224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:35.200253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24012fdb-c027-4177-b30c-f093d2ef30e0 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:35.215929 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-42000b20-f687-409d-ae34-1f3f08427224 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:35.217213 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-42000b20-f687-409d-ae34-1f3f08427224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:35.244899 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27428b-34ac-45d9-af27-621767a29895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:35.245607 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27428b-34ac-45d9-af27-621767a29895 None admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:35.245900 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27428b-34ac-45d9-af27-621767a29895 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:35.246182 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de27428b-34ac-45d9-af27-621767a29895 None admin] PATH_INFO: `/v3/domains/default/users/4fb0c392df554f5dbd68f6861be954a1/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:35.248298 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de27428b-34ac-45d9-af27-621767a29895 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=4fb0c392df554f5dbd68f6861be954a1, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:35.249925 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de27428b-34ac-45d9-af27-621767a29895 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:35.288572 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 19/36] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:34:35 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:34:35.289412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42000b20-f687-409d-ae34-1f3f08427224 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:35.330748 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:35.355204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 18/37] 10.222.0.115 () {58 vars in 1366 bytes} [Tue Aug 2 10:34:35 2022] PUT /identity/v3/domains/default/users/4fb0c392df554f5dbd68f6861be954a1/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:34:35.356267 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27428b-34ac-45d9-af27-621767a29895 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:35.415419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42000b20-f687-409d-ae34-1f3f08427224 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:35.426274 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:35.463041 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2628ca-ca77-4f17-9360-751e6d71ae9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:35.463733 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2628ca-ca77-4f17-9360-751e6d71ae9b None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:35.464016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2628ca-ca77-4f17-9360-751e6d71ae9b None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:35.464356 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2628ca-ca77-4f17-9360-751e6d71ae9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:35.482138 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2628ca-ca77-4f17-9360-751e6d71ae9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:35.483680 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2628ca-ca77-4f17-9360-751e6d71ae9b None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:35.488953 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de27428b-34ac-45d9-af27-621767a29895 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:35.501789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-ff2628ca-ca77-4f17-9360-751e6d71ae9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:34:35.501789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:34:35.501789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:34:35.501789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:34:35.501789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:34:35.501789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:34:35.501789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:34:35.501789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:34:35.501789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:34:35.501789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:34:35.501789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:34:35.501789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:34:35.501789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:34:35.501789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:34:35.501789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:34:35.503197 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:34:35.503197 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:34:35.503197 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:34:35.503197 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:34:35.503197 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:34:35.503197 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:34:35.503197 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:34:35.503197 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:34:35.503197 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:34:35.503197 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:34:35.503197 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:34:35.503197 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:34:35.503197 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:34:35.503197 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:34:35.503197 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:34:35.503197 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:34:35.503197 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:34:35.504080 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:34:35.504080 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:34:35.504080 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:34:35.504080 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:34:35.504080 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 20/38] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:34:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:34:35.509501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291a9e8-ae4d-4b46-84f4-e1c16b565719 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:35.509501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291a9e8-ae4d-4b46-84f4-e1c16b565719 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:35.509501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2291a9e8-ae4d-4b46-84f4-e1c16b565719 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:35.509501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 21/39] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:34:35 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:34:35.509501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291a9e8-ae4d-4b46-84f4-e1c16b565719 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:35.545806 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4482ab6-eff7-49ae-9a8d-20a3e8147f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:35.546350 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4482ab6-eff7-49ae-9a8d-20a3e8147f84 None admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:35.546635 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4482ab6-eff7-49ae-9a8d-20a3e8147f84 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:35.546913 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4482ab6-eff7-49ae-9a8d-20a3e8147f84 None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:35.547713 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4482ab6-eff7-49ae-9a8d-20a3e8147f84 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:35.549100 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4482ab6-eff7-49ae-9a8d-20a3e8147f84 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:35.606195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:35.627221 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 19/40] 10.222.0.115 () {60 vars in 1136 bytes} [Tue Aug 2 10:34:35 2022] POST /identity/v3/roles => generated 226 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:34:35.627221 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4482ab6-eff7-49ae-9a8d-20a3e8147f84 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:35.627933 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2291a9e8-ae4d-4b46-84f4-e1c16b565719 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:35.686907 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1093c44-db52-4644-8342-05d1136954fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:35.688694 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1093c44-db52-4644-8342-05d1136954fc None admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:35.689153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1093c44-db52-4644-8342-05d1136954fc None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:35.689669 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1093c44-db52-4644-8342-05d1136954fc None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:35.691362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1093c44-db52-4644-8342-05d1136954fc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:35.693788 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1093c44-db52-4644-8342-05d1136954fc None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:35.765146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4482ab6-eff7-49ae-9a8d-20a3e8147f84 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:35.783179 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:35.811368 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd5cea-62de-44bb-b48b-4e8338598a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:35.811911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd5cea-62de-44bb-b48b-4e8338598a69 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:35.812197 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd5cea-62de-44bb-b48b-4e8338598a69 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:35.812475 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd5cea-62de-44bb-b48b-4e8338598a69 None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:35.813182 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd5cea-62de-44bb-b48b-4e8338598a69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:34:35.813488 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd5cea-62de-44bb-b48b-4e8338598a69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:35.814876 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd5cea-62de-44bb-b48b-4e8338598a69 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:35.838232 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 20/41] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:34:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:35.839433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9f557-bffb-48ee-a907-cc8103051e9d None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:35.839846 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9f557-bffb-48ee-a907-cc8103051e9d None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:35.840160 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9f557-bffb-48ee-a907-cc8103051e9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:35.860202 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-bad9f557-bffb-48ee-a907-cc8103051e9d None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:35.861405 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-bad9f557-bffb-48ee-a907-cc8103051e9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:35.877462 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 22/42] 10.222.0.115 () {60 vars in 1136 bytes} [Tue Aug 2 10:34:35 2022] POST /identity/v3/roles => generated 224 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:34:35.878866 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1093c44-db52-4644-8342-05d1136954fc None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:35.963353 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 21/43] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:34:35 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:34:35.987906 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad9f557-bffb-48ee-a907-cc8103051e9d None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:36.018308 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1093c44-db52-4644-8342-05d1136954fc None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:36.066908 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca13e8-9d6c-4a73-bc4c-d977fdd5d76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:36.067662 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca13e8-9d6c-4a73-bc4c-d977fdd5d76b None admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:36.068241 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca13e8-9d6c-4a73-bc4c-d977fdd5d76b None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:36.068661 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ca13e8-9d6c-4a73-bc4c-d977fdd5d76b None admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:36.069472 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca13e8-9d6c-4a73-bc4c-d977fdd5d76b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:36.071249 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ca13e8-9d6c-4a73-bc4c-d977fdd5d76b None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:36.112458 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad9f557-bffb-48ee-a907-cc8103051e9d None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:36.116157 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:36.139843 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:36.148596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 23/44] 10.222.0.115 () {60 vars in 1146 bytes} [Tue Aug 2 10:34:35 2022] POST /identity/v3/projects => generated 302 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:34:36.153484 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ea795-8e9a-485f-8a2f-fd546ae5659d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:36.154246 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ea795-8e9a-485f-8a2f-fd546ae5659d None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:36.154545 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ea795-8e9a-485f-8a2f-fd546ae5659d None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:36.155336 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ea795-8e9a-485f-8a2f-fd546ae5659d None admin] PATH_INFO: `/v3/domains/default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:36.157042 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ea795-8e9a-485f-8a2f-fd546ae5659d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:36.159376 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ea795-8e9a-485f-8a2f-fd546ae5659d None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:36.161856 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 22/45] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:34:35 2022] GET /identity/v3/domains/default => generated 200 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:34:36.167902 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca13e8-9d6c-4a73-bc4c-d977fdd5d76b None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:36.298251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ca13e8-9d6c-4a73-bc4c-d977fdd5d76b None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:36.341956 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e96fcbc-5aa9-4b06-8313-83607ef02b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:36.342511 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96fcbc-5aa9-4b06-8313-83607ef02b97 None admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:36.342932 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96fcbc-5aa9-4b06-8313-83607ef02b97 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:36.343132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e96fcbc-5aa9-4b06-8313-83607ef02b97 None admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:36.343795 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96fcbc-5aa9-4b06-8313-83607ef02b97 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:36.345258 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e96fcbc-5aa9-4b06-8313-83607ef02b97 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:36.425680 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 24/46] 10.222.0.115 () {60 vars in 1146 bytes} [Tue Aug 2 10:34:36 2022] POST /identity/v3/projects => generated 313 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:34:37.475120 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e128315-77c5-48ec-b78a-9369a6093666 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:37.475590 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e128315-77c5-48ec-b78a-9369a6093666 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:37.475678 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e128315-77c5-48ec-b78a-9369a6093666 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:37.477179 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 23/47] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:34:37 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:34:37.482390 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fd563-46a0-4da2-a176-aa7e4d2c8ea9 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:37.482803 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fd563-46a0-4da2-a176-aa7e4d2c8ea9 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:37.482957 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61fd563-46a0-4da2-a176-aa7e4d2c8ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:37.503425 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-f61fd563-46a0-4da2-a176-aa7e4d2c8ea9 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:37.505427 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-f61fd563-46a0-4da2-a176-aa7e4d2c8ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:37.553636 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:37.576629 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 25/48] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:34:37 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:34:37.600960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e128315-77c5-48ec-b78a-9369a6093666 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:37.705540 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e128315-77c5-48ec-b78a-9369a6093666 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:37.744362 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92d325-c3e0-492b-aeb1-979f281cf978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:37.744936 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92d325-c3e0-492b-aeb1-979f281cf978 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:37.745226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92d325-c3e0-492b-aeb1-979f281cf978 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:37.745507 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b92d325-c3e0-492b-aeb1-979f281cf978 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:37.763764 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b92d325-c3e0-492b-aeb1-979f281cf978 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:34:37.776275 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-1b92d325-c3e0-492b-aeb1-979f281cf978 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:34:37.776275 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:34:37.776275 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:34:37.776275 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:34:37.776275 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:34:37.776275 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:34:37.776275 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:34:37.776275 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:34:37.776275 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:34:37.776275 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:34:37.776275 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:34:37.776275 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:34:37.776275 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:34:37.776275 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:34:37.776275 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:34:37.776275 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:34:37.777144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:34:37.777144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:34:37.777144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:34:37.777144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:34:37.777144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:34:37.777144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:34:37.777144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:34:37.777144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:34:37.777144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:34:37.777144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:34:37.777144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:34:37.777144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:34:37.777144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:34:37.777144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:34:37.777144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:34:37.777144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:34:37.777914 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:34:37.777914 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:34:37.777914 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:34:37.777914 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:34:37.777914 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:34:37.777914 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:34:37.777914 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:34:37.777914 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:34:37.777914 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:34:37.777914 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:34:37.777914 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:34:37.777914 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:34:37.777914 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 24/49] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:34:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:34:37.783362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fd563-46a0-4da2-a176-aa7e4d2c8ea9 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:37.886833 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61fd563-46a0-4da2-a176-aa7e4d2c8ea9 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:37.930135 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f50f9-97c0-4821-896b-cea998d1cdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:37.930762 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f50f9-97c0-4821-896b-cea998d1cdb4 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:37.931065 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f50f9-97c0-4821-896b-cea998d1cdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:37.931350 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02f50f9-97c0-4821-896b-cea998d1cdb4 None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:37.932076 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f50f9-97c0-4821-896b-cea998d1cdb4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:34:37.932379 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f50f9-97c0-4821-896b-cea998d1cdb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:37.933785 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02f50f9-97c0-4821-896b-cea998d1cdb4 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:37.942734 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 26/50] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:34:37 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:34:37.948286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92d325-c3e0-492b-aeb1-979f281cf978 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:37.983082 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:38.055673 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b92d325-c3e0-492b-aeb1-979f281cf978 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:38.098923 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa3e51-dd9d-4780-b592-a816c2235271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:38.099719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa3e51-dd9d-4780-b592-a816c2235271 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:38.100071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa3e51-dd9d-4780-b592-a816c2235271 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:38.100419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fa3e51-dd9d-4780-b592-a816c2235271 None admin] PATH_INFO: `/v3/users/4fb0c392df554f5dbd68f6861be954a1` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:38.101697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fa3e51-dd9d-4780-b592-a816c2235271 None admin] RBAC: Authorizing `identity:get_user(user_id=4fb0c392df554f5dbd68f6861be954a1)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:38.103317 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fa3e51-dd9d-4780-b592-a816c2235271 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:38.105063 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 25/51] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:34:37 2022] GET /identity/v3/users/4fb0c392df554f5dbd68f6861be954a1 => generated 248 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:38.112115 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f50f9-97c0-4821-896b-cea998d1cdb4 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:38.162066 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:38.231274 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02f50f9-97c0-4821-896b-cea998d1cdb4 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:38.266789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a39c4d-e628-4d80-93f0-f178bb090eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:38.267467 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a39c4d-e628-4d80-93f0-f178bb090eba None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:38.267755 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a39c4d-e628-4d80-93f0-f178bb090eba None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:38.268033 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a39c4d-e628-4d80-93f0-f178bb090eba None admin] PATH_INFO: `/v3/domains/default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:38.270055 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a39c4d-e628-4d80-93f0-f178bb090eba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:38.272625 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a39c4d-e628-4d80-93f0-f178bb090eba None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:38.275420 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 27/52] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:34:38 2022] GET /identity/v3/domains/default => generated 200 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:34:38.281348 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa3e51-dd9d-4780-b592-a816c2235271 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:38.389435 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:38.392314 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fa3e51-dd9d-4780-b592-a816c2235271 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:38.428535 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200e5d2c-6f4c-46f5-8837-069f44f4b0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:38.429148 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e5d2c-6f4c-46f5-8837-069f44f4b0e5 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:38.429426 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e5d2c-6f4c-46f5-8837-069f44f4b0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:38.429700 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200e5d2c-6f4c-46f5-8837-069f44f4b0e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:38.430496 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e5d2c-6f4c-46f5-8837-069f44f4b0e5 None admin] RBAC: Adding query filter params (role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac, scope.domain.id=default, user.id=4fb0c392df554f5dbd68f6861be954a1) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:34:38.430801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e5d2c-6f4c-46f5-8837-069f44f4b0e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:38.432333 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200e5d2c-6f4c-46f5-8837-069f44f4b0e5 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:38.441655 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 26/53] 10.222.0.115 () {56 vars in 1327 bytes} [Tue Aug 2 10:34:38 2022] GET /identity/v3/role_assignments?user.id=4fb0c392df554f5dbd68f6861be954a1&scope.domain.id=default&role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 532 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:38.537953 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9b29a-0453-4e5a-a22f-11d4d74fd00a None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:38.538493 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9b29a-0453-4e5a-a22f-11d4d74fd00a None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:38.538602 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de9b29a-0453-4e5a-a22f-11d4d74fd00a None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:38.540192 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 28/54] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:34:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:34:38.550595 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69850c47-e2f0-49a6-b64b-4de286af4af5 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:38.551364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69850c47-e2f0-49a6-b64b-4de286af4af5 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:38.551487 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69850c47-e2f0-49a6-b64b-4de286af4af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:38.572762 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-69850c47-e2f0-49a6-b64b-4de286af4af5 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:38.574129 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-69850c47-e2f0-49a6-b64b-4de286af4af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:38.657528 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 27/55] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:34:38 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:34:38.680885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9b29a-0453-4e5a-a22f-11d4d74fd00a None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:38.796002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de9b29a-0453-4e5a-a22f-11d4d74fd00a None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:38.806879 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:38.832709 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de916b99-5803-47e8-835b-55ca9435c77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:38.833600 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de916b99-5803-47e8-835b-55ca9435c77e None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:38.833897 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de916b99-5803-47e8-835b-55ca9435c77e None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:38.834211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de916b99-5803-47e8-835b-55ca9435c77e None admin] PATH_INFO: `/v3/domains/default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:38.835537 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de916b99-5803-47e8-835b-55ca9435c77e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:38.837323 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de916b99-5803-47e8-835b-55ca9435c77e None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:38.839301 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 29/56] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:34:38 2022] GET /identity/v3/domains/default => generated 200 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:38.845026 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69850c47-e2f0-49a6-b64b-4de286af4af5 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:38.943744 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:38.952301 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69850c47-e2f0-49a6-b64b-4de286af4af5 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:38.991299 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d80c38f-7ccc-4ede-ba32-e87b7e63874a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:38.991913 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80c38f-7ccc-4ede-ba32-e87b7e63874a None admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:38.992195 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80c38f-7ccc-4ede-ba32-e87b7e63874a None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:38.992465 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d80c38f-7ccc-4ede-ba32-e87b7e63874a None admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:38.993154 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d80c38f-7ccc-4ede-ba32-e87b7e63874a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:38.994660 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d80c38f-7ccc-4ede-ba32-e87b7e63874a None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:39.118968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 28/57] 10.222.0.115 () {60 vars in 1145 bytes} [Tue Aug 2 10:34:38 2022] POST /identity/v3/projects => generated 299 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:34:41.208242 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a30f61-bf5e-416f-9d6a-393852163607 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:41.208762 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a30f61-bf5e-416f-9d6a-393852163607 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:41.208851 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a30f61-bf5e-416f-9d6a-393852163607 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:41.210274 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 30/58] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:34:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:34:41.215778 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edcc20-a1f7-44dc-a6d7-b2285741b919 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:41.216291 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edcc20-a1f7-44dc-a6d7-b2285741b919 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:41.216403 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65edcc20-a1f7-44dc-a6d7-b2285741b919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:41.234246 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-65edcc20-a1f7-44dc-a6d7-b2285741b919 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:41.235474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-65edcc20-a1f7-44dc-a6d7-b2285741b919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:41.316653 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 29/59] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:34:41 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:34:41.340810 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a30f61-bf5e-416f-9d6a-393852163607 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:41.474996 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a30f61-bf5e-416f-9d6a-393852163607 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:41.515849 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e067d55-c4a1-46cd-b9ce-df00b6289fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:41.516553 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e067d55-c4a1-46cd-b9ce-df00b6289fa0 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:41.516851 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e067d55-c4a1-46cd-b9ce-df00b6289fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:41.517128 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e067d55-c4a1-46cd-b9ce-df00b6289fa0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:41.518381 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e067d55-c4a1-46cd-b9ce-df00b6289fa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:41.519996 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e067d55-c4a1-46cd-b9ce-df00b6289fa0 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:41.521608 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 31/60] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:34:41 2022] GET /identity/v3/domains/default => generated 200 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:41.527473 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65edcc20-a1f7-44dc-a6d7-b2285741b919 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:41.609712 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:41.634239 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65edcc20-a1f7-44dc-a6d7-b2285741b919 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:41.688237 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5a377-ea1f-49bb-a1bf-9af7fcb78dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:41.688871 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5a377-ea1f-49bb-a1bf-9af7fcb78dcb None admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:41.689164 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5a377-ea1f-49bb-a1bf-9af7fcb78dcb None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:41.689542 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5a377-ea1f-49bb-a1bf-9af7fcb78dcb None admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:41.690760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5a377-ea1f-49bb-a1bf-9af7fcb78dcb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:41.692606 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5a377-ea1f-49bb-a1bf-9af7fcb78dcb None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:41.808088 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 30/61] 10.222.0.115 () {60 vars in 1137 bytes} [Tue Aug 2 10:34:41 2022] POST /identity/v3/users => generated 276 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:34:45.485685 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f31c42-68a5-473a-9cfc-5bf79822f389 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:45.486285 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f31c42-68a5-473a-9cfc-5bf79822f389 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:45.486285 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f31c42-68a5-473a-9cfc-5bf79822f389 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:45.489005 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 32/62] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:34:45 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:34:45.495956 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97cf46-0985-43b4-8d65-b76826c9e80a None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:45.496578 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97cf46-0985-43b4-8d65-b76826c9e80a None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:45.496578 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a97cf46-0985-43b4-8d65-b76826c9e80a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:45.510561 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ceb44-d8f5-4c79-82e9-60fed65569d2 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:45.511207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ceb44-d8f5-4c79-82e9-60fed65569d2 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:45.511694 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ceb44-d8f5-4c79-82e9-60fed65569d2 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:45.513404 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 33/63] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:34:45 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:34:45.519529 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219447a6-4af8-4a51-9915-44f3cca232d8 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:45.519829 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219447a6-4af8-4a51-9915-44f3cca232d8 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:45.520093 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219447a6-4af8-4a51-9915-44f3cca232d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:45.535182 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-5a97cf46-0985-43b4-8d65-b76826c9e80a None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:45.536789 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-5a97cf46-0985-43b4-8d65-b76826c9e80a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:45.552216 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-219447a6-4af8-4a51-9915-44f3cca232d8 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:45.553544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-219447a6-4af8-4a51-9915-44f3cca232d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:45.628566 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:45.666718 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 31/64] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:34:45 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:34:45.685899 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 34/65] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:34:45 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:34:45.698909 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97cf46-0985-43b4-8d65-b76826c9e80a None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:45.717790 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219447a6-4af8-4a51-9915-44f3cca232d8 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:45.809860 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:45.861733 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a97cf46-0985-43b4-8d65-b76826c9e80a None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:45.912488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219447a6-4af8-4a51-9915-44f3cca232d8 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:45.917215 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c250b27-16ca-40e0-86b1-bfd5f3af7533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:45.917950 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c250b27-16ca-40e0-86b1-bfd5f3af7533 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:45.918231 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c250b27-16ca-40e0-86b1-bfd5f3af7533 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:45.918501 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c250b27-16ca-40e0-86b1-bfd5f3af7533 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:45.930453 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c250b27-16ca-40e0-86b1-bfd5f3af7533 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:34:45.936001 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-5c250b27-16ca-40e0-86b1-bfd5f3af7533 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:34:45.936001 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:34:45.936001 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:34:45.936001 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:34:45.936001 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:34:45.936001 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:34:45.936001 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:34:45.936001 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:34:45.936001 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:34:45.936001 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:34:45.936001 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:34:45.936001 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:34:45.936001 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:34:45.936001 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:34:45.936001 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:34:45.936001 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:34:45.937999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:34:45.937999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:34:45.937999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:34:45.937999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:34:45.937999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:34:45.937999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:34:45.937999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:34:45.937999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:34:45.937999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:34:45.937999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:34:45.937999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:34:45.937999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:34:45.937999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:34:45.937999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:34:45.937999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:34:45.937999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:34:45.939757 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:34:45.939757 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:34:45.939757 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:34:45.939757 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:34:45.939757 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:34:45.939757 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:34:45.939757 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:34:45.939757 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:34:45.939757 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:34:45.939757 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:34:45.939757 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:34:45.939757 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:34:45.939757 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 32/66] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:34:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:34:45.939757 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664642b-86e8-4974-9441-e76ddbbaa9c4 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:45.947271 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664642b-86e8-4974-9441-e76ddbbaa9c4 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:45.947271 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664642b-86e8-4974-9441-e76ddbbaa9c4 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:45.947271 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 33/67] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:34:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:34:45.947271 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70856c59-3e18-4b81-a3f6-9361c880ff6b None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:45.947271 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70856c59-3e18-4b81-a3f6-9361c880ff6b None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:45.947271 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70856c59-3e18-4b81-a3f6-9361c880ff6b None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:45.947271 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 34/68] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:34:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:34:45.950151 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70856c59-3e18-4b81-a3f6-9361c880ff6b None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:45.967956 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab4ab0e-f5c0-4714-bb0a-810bfd2f5ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:45.968960 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4ab0e-f5c0-4714-bb0a-810bfd2f5ff0 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:45.969438 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4ab0e-f5c0-4714-bb0a-810bfd2f5ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:45.969738 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab4ab0e-f5c0-4714-bb0a-810bfd2f5ff0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:45.981775 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab4ab0e-f5c0-4714-bb0a-810bfd2f5ff0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:34:45.987452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-dab4ab0e-f5c0-4714-bb0a-810bfd2f5ff0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:34:45.987452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:34:45.987452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:34:45.987452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:34:45.987452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:34:45.987452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:34:45.987452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:34:45.987452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:34:45.987452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:34:45.987452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:34:45.987452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:34:45.987452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:34:45.987452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:34:45.987452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:34:45.987452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:34:45.987452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:34:45.988977 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:34:45.988977 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:34:45.988977 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:34:45.988977 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:34:45.988977 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:34:45.988977 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:34:45.988977 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:34:45.988977 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:34:45.988977 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:34:45.988977 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:34:45.988977 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:34:45.988977 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:34:45.988977 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:34:45.988977 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:34:45.988977 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:34:45.988977 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:34:45.989916 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:34:45.989916 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:34:45.989916 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:34:45.989916 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:34:45.989916 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:34:45.989916 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:34:45.989916 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:34:45.989916 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:34:45.989916 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:34:45.989916 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:34:45.989916 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:34:45.989916 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:34:45.989916 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 35/69] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:34:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:34:45.991103 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ca4e1-358d-4c31-b633-7403298bba77 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:45.991198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ca4e1-358d-4c31-b633-7403298bba77 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:45.991514 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796ca4e1-358d-4c31-b633-7403298bba77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:46.018794 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-796ca4e1-358d-4c31-b633-7403298bba77 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:46.020155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-796ca4e1-358d-4c31-b633-7403298bba77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:46.062644 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:46.106835 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70856c59-3e18-4b81-a3f6-9361c880ff6b None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:46.120808 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 36/70] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:34:45 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:34:46.122893 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5d433-eee0-44ec-b48d-9e19b9a60669 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:46.123358 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5d433-eee0-44ec-b48d-9e19b9a60669 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:46.124201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c5d433-eee0-44ec-b48d-9e19b9a60669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:46.133079 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:46.145599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dceb07-d50c-4031-a3bf-b94e1268b1e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:46.151716 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dceb07-d50c-4031-a3bf-b94e1268b1e4 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:46.151716 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dceb07-d50c-4031-a3bf-b94e1268b1e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:46.151716 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84dceb07-d50c-4031-a3bf-b94e1268b1e4 None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:46.151716 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dceb07-d50c-4031-a3bf-b94e1268b1e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:34:46.151716 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dceb07-d50c-4031-a3bf-b94e1268b1e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:46.151716 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84dceb07-d50c-4031-a3bf-b94e1268b1e4 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:46.152576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-03c5d433-eee0-44ec-b48d-9e19b9a60669 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:46.152576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-03c5d433-eee0-44ec-b48d-9e19b9a60669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:46.164772 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 35/71] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:34:45 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:46.166299 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dceb07-d50c-4031-a3bf-b94e1268b1e4 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:46.234362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 37/72] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:34:46 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:34:46.235559 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22dd0a-58b6-4aaa-8eb7-e91a79630572 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:46.235847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22dd0a-58b6-4aaa-8eb7-e91a79630572 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:46.236125 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a22dd0a-58b6-4aaa-8eb7-e91a79630572 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:46.238081 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 38/73] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:34:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:34:46.238789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22dd0a-58b6-4aaa-8eb7-e91a79630572 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:46.291004 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84dceb07-d50c-4031-a3bf-b94e1268b1e4 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:46.327783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275dcff-8b72-4564-a2b9-b718cb6a1d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:46.328401 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275dcff-8b72-4564-a2b9-b718cb6a1d9e None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:46.328674 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275dcff-8b72-4564-a2b9-b718cb6a1d9e None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:46.328952 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b275dcff-8b72-4564-a2b9-b718cb6a1d9e None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:46.329669 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b275dcff-8b72-4564-a2b9-b718cb6a1d9e None admin] RBAC: Adding query filter params (name=member) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:34:46.329970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b275dcff-8b72-4564-a2b9-b718cb6a1d9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:46.331784 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b275dcff-8b72-4564-a2b9-b718cb6a1d9e None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:46.340415 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 36/74] 10.222.0.115 () {56 vars in 1106 bytes} [Tue Aug 2 10:34:46 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:46.341212 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275dcff-8b72-4564-a2b9-b718cb6a1d9e None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:46.359098 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:46.367710 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a22dd0a-58b6-4aaa-8eb7-e91a79630572 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:46.398150 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:46.407924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9dfc8-6401-4629-a3a5-bec618c89210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:46.415643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9dfc8-6401-4629-a3a5-bec618c89210 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:46.415643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9dfc8-6401-4629-a3a5-bec618c89210 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:46.415643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9dfc8-6401-4629-a3a5-bec618c89210 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:46.419927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d9dfc8-6401-4629-a3a5-bec618c89210 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:34:46.425776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-25d9dfc8-6401-4629-a3a5-bec618c89210 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:34:46.425776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:34:46.425776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:34:46.425776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:34:46.425776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:34:46.425776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:34:46.425776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:34:46.425776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:34:46.425776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:34:46.425776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:34:46.425776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:34:46.425776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:34:46.425776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:34:46.425776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:34:46.425776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:34:46.425776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:34:46.426691 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:34:46.426691 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:34:46.426691 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:34:46.426691 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:34:46.426691 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:34:46.426691 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:34:46.426691 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:34:46.426691 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:34:46.426691 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:34:46.426691 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:34:46.426691 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:34:46.426691 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:34:46.426691 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:34:46.426691 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:34:46.426691 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:34:46.426691 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:34:46.427564 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:34:46.427564 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:34:46.427564 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:34:46.427564 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:34:46.427564 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:34:46.427564 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:34:46.427564 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:34:46.427564 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:34:46.427564 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:34:46.427564 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:34:46.427564 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:34:46.427564 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:34:46.427564 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 39/75] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:34:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:34:46.428701 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d87811-fae5-4bf8-a387-7f45ba50878d None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:46.428701 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d87811-fae5-4bf8-a387-7f45ba50878d None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:46.428838 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d87811-fae5-4bf8-a387-7f45ba50878d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:46.450282 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-79d87811-fae5-4bf8-a387-7f45ba50878d None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:46.451534 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-79d87811-fae5-4bf8-a387-7f45ba50878d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:46.464658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b275dcff-8b72-4564-a2b9-b718cb6a1d9e None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:46.500647 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739ffd9-ceea-4f0a-907b-9cf19c92903f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:46.501287 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739ffd9-ceea-4f0a-907b-9cf19c92903f None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:46.501451 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739ffd9-ceea-4f0a-907b-9cf19c92903f None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:46.501726 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739ffd9-ceea-4f0a-907b-9cf19c92903f None admin] PATH_INFO: `/v3/users/4fb0c392df554f5dbd68f6861be954a1` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:46.503864 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3739ffd9-ceea-4f0a-907b-9cf19c92903f None admin] RBAC: Authorizing `identity:get_user(user_id=4fb0c392df554f5dbd68f6861be954a1)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:46.505473 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3739ffd9-ceea-4f0a-907b-9cf19c92903f None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:46.507159 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 37/76] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:34:46 2022] GET /identity/v3/users/4fb0c392df554f5dbd68f6861be954a1 => generated 248 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:46.507932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739ffd9-ceea-4f0a-907b-9cf19c92903f None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:46.532634 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 40/77] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:34:46 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:34:46.533571 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d87811-fae5-4bf8-a387-7f45ba50878d None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:46.650827 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739ffd9-ceea-4f0a-907b-9cf19c92903f None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:46.652636 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:46.667976 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:46.669053 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d87811-fae5-4bf8-a387-7f45ba50878d None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:46.694106 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d1f00-3d21-4724-bfa0-d9fa13222db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:46.695803 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d1f00-3d21-4724-bfa0-d9fa13222db9 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:46.695803 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d1f00-3d21-4724-bfa0-d9fa13222db9 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:46.695803 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8d1f00-3d21-4724-bfa0-d9fa13222db9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:46.708513 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8d1f00-3d21-4724-bfa0-d9fa13222db9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:34:46.710245 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1274c-a2d2-4593-8d27-a145333f604e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:46.710893 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1274c-a2d2-4593-8d27-a145333f604e None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:46.711224 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1274c-a2d2-4593-8d27-a145333f604e None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:46.711581 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1274c-a2d2-4593-8d27-a145333f604e None admin] PATH_INFO: `/v3/users/6636e4878e5846e598b80dcba8afbeef` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:46.713605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-4a8d1f00-3d21-4724-bfa0-d9fa13222db9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:34:46.713605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:34:46.713605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:34:46.713605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:34:46.713605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:34:46.713605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:34:46.713605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:34:46.713605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:34:46.713605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:34:46.713605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:34:46.713605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:34:46.713605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:34:46.713605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:34:46.713605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:34:46.713605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:34:46.713605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:34:46.715104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:34:46.715104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:34:46.715104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:34:46.715104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:34:46.715104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:34:46.715104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:34:46.715104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:34:46.715104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:34:46.715104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:34:46.715104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:34:46.715104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:34:46.715104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:34:46.715104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:34:46.715104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:34:46.715104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:34:46.715104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:34:46.717328 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:34:46.717328 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:34:46.717328 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:34:46.717328 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:34:46.717328 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:34:46.717328 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:34:46.717328 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:34:46.717328 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:34:46.717328 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:34:46.717328 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:34:46.717328 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:34:46.717328 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:34:46.717328 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 38/78] 10.222.0.115 () {56 vars in 1119 bytes} [Tue Aug 2 10:34:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:34:46.717328 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d1f00-3d21-4724-bfa0-d9fa13222db9 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:46.742515 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c1274c-a2d2-4593-8d27-a145333f604e None admin] RBAC: Authorizing `identity:get_user(user_id=6636e4878e5846e598b80dcba8afbeef)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:46.744067 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c1274c-a2d2-4593-8d27-a145333f604e None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:46.747570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 41/79] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:34:46 2022] GET /identity/v3/users/6636e4878e5846e598b80dcba8afbeef => generated 276 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:46.748407 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1274c-a2d2-4593-8d27-a145333f604e None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:46.847293 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8d1f00-3d21-4724-bfa0-d9fa13222db9 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:46.885205 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd25ff-846f-45db-8cc0-bff09341b2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:46.885535 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1274c-a2d2-4593-8d27-a145333f604e None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:46.885822 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd25ff-846f-45db-8cc0-bff09341b2c4 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:46.886107 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd25ff-846f-45db-8cc0-bff09341b2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:46.886385 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cd25ff-846f-45db-8cc0-bff09341b2c4 None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:46.887140 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd25ff-846f-45db-8cc0-bff09341b2c4 None admin] RBAC: Adding query filter params (name=member) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:34:46.887457 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd25ff-846f-45db-8cc0-bff09341b2c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:46.888866 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cd25ff-846f-45db-8cc0-bff09341b2c4 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:46.909430 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 39/80] 10.222.0.115 () {56 vars in 1106 bytes} [Tue Aug 2 10:34:46 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:46.909430 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd25ff-846f-45db-8cc0-bff09341b2c4 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:46.924310 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415b0b7-0001-4623-9d88-a767b383a392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:46.925023 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415b0b7-0001-4623-9d88-a767b383a392 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:46.925309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415b0b7-0001-4623-9d88-a767b383a392 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:46.925590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b415b0b7-0001-4623-9d88-a767b383a392 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:46.941668 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:46.944969 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:46.997867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b415b0b7-0001-4623-9d88-a767b383a392 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:46.999399 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b415b0b7-0001-4623-9d88-a767b383a392 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:47.015312 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cd25ff-846f-45db-8cc0-bff09341b2c4 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:47.051131 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146501c-3fa4-4b1c-b72b-d13179b577f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:47.051793 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146501c-3fa4-4b1c-b72b-d13179b577f0 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:47.052060 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146501c-3fa4-4b1c-b72b-d13179b577f0 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:47.052342 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4146501c-3fa4-4b1c-b72b-d13179b577f0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:47.054433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146501c-3fa4-4b1c-b72b-d13179b577f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:47.055111 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4146501c-3fa4-4b1c-b72b-d13179b577f0 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:47.056763 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 40/81] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:34:46 2022] GET /identity/v3/domains/default => generated 200 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:47.057609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146501c-3fa4-4b1c-b72b-d13179b577f0 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:47.119214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 42/82] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:34:46 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:34:47.124029 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415b0b7-0001-4623-9d88-a767b383a392 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:47.173219 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:47.197181 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4146501c-3fa4-4b1c-b72b-d13179b577f0 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:47.199611 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:47.231706 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94196b3e-b6f1-44c7-bdf1-6dafa3d96251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:47.232318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94196b3e-b6f1-44c7-bdf1-6dafa3d96251 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:47.232600 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94196b3e-b6f1-44c7-bdf1-6dafa3d96251 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:47.232872 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94196b3e-b6f1-44c7-bdf1-6dafa3d96251 None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:47.233613 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94196b3e-b6f1-44c7-bdf1-6dafa3d96251 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:34:47.233936 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94196b3e-b6f1-44c7-bdf1-6dafa3d96251 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:47.235339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94196b3e-b6f1-44c7-bdf1-6dafa3d96251 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:47.237508 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b415b0b7-0001-4623-9d88-a767b383a392 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:47.245015 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 41/83] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:34:47 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:47.245667 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94196b3e-b6f1-44c7-bdf1-6dafa3d96251 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:47.280069 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a4df6-1b30-4e72-88ee-c0cf7a0c0d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:47.280694 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a4df6-1b30-4e72-88ee-c0cf7a0c0d76 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:47.280987 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a4df6-1b30-4e72-88ee-c0cf7a0c0d76 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:47.281263 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a4df6-1b30-4e72-88ee-c0cf7a0c0d76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:47.355957 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94196b3e-b6f1-44c7-bdf1-6dafa3d96251 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:47.358479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a4df6-1b30-4e72-88ee-c0cf7a0c0d76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:47.360028 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a4df6-1b30-4e72-88ee-c0cf7a0c0d76 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:47.391381 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f756d9-9d5d-4076-9993-8ceab88f1b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:47.391943 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f756d9-9d5d-4076-9993-8ceab88f1b82 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:47.392269 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f756d9-9d5d-4076-9993-8ceab88f1b82 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:47.392503 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f756d9-9d5d-4076-9993-8ceab88f1b82 None admin] PATH_INFO: `/v3/users/6636e4878e5846e598b80dcba8afbeef` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:47.393932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f756d9-9d5d-4076-9993-8ceab88f1b82 None admin] RBAC: Authorizing `identity:get_user(user_id=6636e4878e5846e598b80dcba8afbeef)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:47.395430 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f756d9-9d5d-4076-9993-8ceab88f1b82 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:47.397075 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 42/84] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:34:47 2022] GET /identity/v3/users/6636e4878e5846e598b80dcba8afbeef => generated 276 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:47.400163 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f756d9-9d5d-4076-9993-8ceab88f1b82 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:47.411242 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:47.439554 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:47.471867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 43/85] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:34:47 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:34:47.472819 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a4df6-1b30-4e72-88ee-c0cf7a0c0d76 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:47.516036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f756d9-9d5d-4076-9993-8ceab88f1b82 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:47.552432 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1075ac-1dd1-40e6-af9e-063d70b4ad2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:47.553078 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1075ac-1dd1-40e6-af9e-063d70b4ad2b None admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:47.553343 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1075ac-1dd1-40e6-af9e-063d70b4ad2b None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:47.553622 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1075ac-1dd1-40e6-af9e-063d70b4ad2b None admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:47.554315 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1075ac-1dd1-40e6-af9e-063d70b4ad2b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:47.555813 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1075ac-1dd1-40e6-af9e-063d70b4ad2b None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:47.582160 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a4df6-1b30-4e72-88ee-c0cf7a0c0d76 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:47.620612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d40ffb0-ac65-4d6e-8030-70e3665d8c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:47.621265 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40ffb0-ac65-4d6e-8030-70e3665d8c88 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:47.621549 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40ffb0-ac65-4d6e-8030-70e3665d8c88 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:47.621829 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d40ffb0-ac65-4d6e-8030-70e3665d8c88 None admin] PATH_INFO: `/v3/projects/459ff8f521284cf8b3ec858bfb1ba630` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:47.623096 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d40ffb0-ac65-4d6e-8030-70e3665d8c88 None admin] RBAC: Authorizing `identity:get_project(project_id=459ff8f521284cf8b3ec858bfb1ba630)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:47.624495 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d40ffb0-ac65-4d6e-8030-70e3665d8c88 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:47.626189 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 44/86] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:34:47 2022] GET /identity/v3/projects/459ff8f521284cf8b3ec858bfb1ba630 => generated 299 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:47.627034 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d40ffb0-ac65-4d6e-8030-70e3665d8c88 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:47.641785 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:47.711177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 43/87] 10.222.0.115 () {60 vars in 1137 bytes} [Tue Aug 2 10:34:47 2022] POST /identity/v3/users => generated 290 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:34:47.712290 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1075ac-1dd1-40e6-af9e-063d70b4ad2b None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:47.763249 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d40ffb0-ac65-4d6e-8030-70e3665d8c88 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:47.814607 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923cbcf-63f5-4719-86c5-db733bf7a1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:47.815580 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923cbcf-63f5-4719-86c5-db733bf7a1e9 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:47.815869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923cbcf-63f5-4719-86c5-db733bf7a1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:47.816152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7923cbcf-63f5-4719-86c5-db733bf7a1e9 None admin] PATH_INFO: `/v3/users/6636e4878e5846e598b80dcba8afbeef` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:47.817559 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7923cbcf-63f5-4719-86c5-db733bf7a1e9 None admin] RBAC: Authorizing `identity:get_user(user_id=6636e4878e5846e598b80dcba8afbeef)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:47.819157 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7923cbcf-63f5-4719-86c5-db733bf7a1e9 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:47.821292 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 45/88] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:34:47 2022] GET /identity/v3/users/6636e4878e5846e598b80dcba8afbeef => generated 276 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:47.821883 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923cbcf-63f5-4719-86c5-db733bf7a1e9 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:47.855790 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:47.870487 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1075ac-1dd1-40e6-af9e-063d70b4ad2b None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:47.919306 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a7f09-c2ad-4943-907a-c80161c7d25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:47.919683 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a7f09-c2ad-4943-907a-c80161c7d25d None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:47.920024 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a7f09-c2ad-4943-907a-c80161c7d25d None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:47.920340 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298a7f09-c2ad-4943-907a-c80161c7d25d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:47.929662 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:47.960815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7923cbcf-63f5-4719-86c5-db733bf7a1e9 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:48.007797 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dfdf2e-c13f-4f09-9be6-4db1aaa98c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:48.008753 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dfdf2e-c13f-4f09-9be6-4db1aaa98c83 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:48.010336 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dfdf2e-c13f-4f09-9be6-4db1aaa98c83 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:48.010336 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dfdf2e-c13f-4f09-9be6-4db1aaa98c83 None admin] PATH_INFO: `/v3/projects/459ff8f521284cf8b3ec858bfb1ba630` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:48.010716 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dfdf2e-c13f-4f09-9be6-4db1aaa98c83 None admin] RBAC: Authorizing `identity:get_project(project_id=459ff8f521284cf8b3ec858bfb1ba630)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:48.012329 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dfdf2e-c13f-4f09-9be6-4db1aaa98c83 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:48.013989 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 46/89] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:34:47 2022] GET /identity/v3/projects/459ff8f521284cf8b3ec858bfb1ba630 => generated 299 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:48.014890 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dfdf2e-c13f-4f09-9be6-4db1aaa98c83 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:48.015012 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a7f09-c2ad-4943-907a-c80161c7d25d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:48.016718 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298a7f09-c2ad-4943-907a-c80161c7d25d None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:48.155787 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:48.156612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dfdf2e-c13f-4f09-9be6-4db1aaa98c83 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:48.174920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 44/90] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:34:47 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:34:48.174920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a7f09-c2ad-4943-907a-c80161c7d25d None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:48.206625 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620fde93-66e0-4ee7-936f-5e98de882771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:48.208487 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fde93-66e0-4ee7-936f-5e98de882771 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:48.209034 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fde93-66e0-4ee7-936f-5e98de882771 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:48.209523 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620fde93-66e0-4ee7-936f-5e98de882771 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:48.210544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620fde93-66e0-4ee7-936f-5e98de882771 None admin] RBAC: Adding query filter params (role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac, scope.project.id=459ff8f521284cf8b3ec858bfb1ba630, user.id=4fb0c392df554f5dbd68f6861be954a1) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:34:48.211094 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620fde93-66e0-4ee7-936f-5e98de882771 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:48.212779 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620fde93-66e0-4ee7-936f-5e98de882771 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:48.223569 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 47/91] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:34:48 2022] GET /identity/v3/role_assignments?user.id=4fb0c392df554f5dbd68f6861be954a1&scope.project.id=459ff8f521284cf8b3ec858bfb1ba630&role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 260 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:48.224755 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620fde93-66e0-4ee7-936f-5e98de882771 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:48.249609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:48.363183 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620fde93-66e0-4ee7-936f-5e98de882771 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:48.416900 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298a7f09-c2ad-4943-907a-c80161c7d25d None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:48.424331 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287d04c-629f-476a-ad65-e1a40fedb882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:48.425383 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287d04c-629f-476a-ad65-e1a40fedb882 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:48.425804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287d04c-629f-476a-ad65-e1a40fedb882 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:48.426298 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5287d04c-629f-476a-ad65-e1a40fedb882 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:48.427860 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287d04c-629f-476a-ad65-e1a40fedb882 None admin] RBAC: Adding query filter params (role.id=33c5141e078844bbb60127f2b26540d7, scope.project.id=459ff8f521284cf8b3ec858bfb1ba630, user.id=6636e4878e5846e598b80dcba8afbeef) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:34:48.428595 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287d04c-629f-476a-ad65-e1a40fedb882 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:48.430720 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5287d04c-629f-476a-ad65-e1a40fedb882 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:48.455267 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 48/92] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:34:48 2022] GET /identity/v3/role_assignments?user.id=6636e4878e5846e598b80dcba8afbeef&scope.project.id=459ff8f521284cf8b3ec858bfb1ba630&role.id=33c5141e078844bbb60127f2b26540d7 => generated 260 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:48.457138 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287d04c-629f-476a-ad65-e1a40fedb882 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:48.491405 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f07c8-3624-46e3-a37d-504f2ea7b24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:48.492126 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f07c8-3624-46e3-a37d-504f2ea7b24c None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:48.492521 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f07c8-3624-46e3-a37d-504f2ea7b24c None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:48.492730 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789f07c8-3624-46e3-a37d-504f2ea7b24c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:48.637428 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:48.665428 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:48.678393 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5287d04c-629f-476a-ad65-e1a40fedb882 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:48.696932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f07c8-3624-46e3-a37d-504f2ea7b24c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:48.698508 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789f07c8-3624-46e3-a37d-504f2ea7b24c None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:48.759402 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b66ced-e0c2-43f4-a9ad-5b517ec92b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:48.760714 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b66ced-e0c2-43f4-a9ad-5b517ec92b58 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:48.761166 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b66ced-e0c2-43f4-a9ad-5b517ec92b58 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:48.761594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b66ced-e0c2-43f4-a9ad-5b517ec92b58 None admin] PATH_INFO: `/v3/projects/009a328c01ce4fed9c5b90fec435b08f` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:48.763141 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b66ced-e0c2-43f4-a9ad-5b517ec92b58 None admin] RBAC: Authorizing `identity:get_project(project_id=009a328c01ce4fed9c5b90fec435b08f)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:48.765528 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b66ced-e0c2-43f4-a9ad-5b517ec92b58 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:48.769787 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 49/93] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:34:48 2022] GET /identity/v3/projects/009a328c01ce4fed9c5b90fec435b08f => generated 313 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:48.778687 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b66ced-e0c2-43f4-a9ad-5b517ec92b58 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:48.955162 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 45/94] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:34:48 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 783 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:34:48.962536 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f07c8-3624-46e3-a37d-504f2ea7b24c None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:48.965374 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b66ced-e0c2-43f4-a9ad-5b517ec92b58 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:49.014764 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2336c5e1-ee0f-47e5-9ec7-2d21550eec71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:49.015708 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336c5e1-ee0f-47e5-9ec7-2d21550eec71 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:49.016005 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336c5e1-ee0f-47e5-9ec7-2d21550eec71 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:49.016296 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2336c5e1-ee0f-47e5-9ec7-2d21550eec71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:49.017405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2336c5e1-ee0f-47e5-9ec7-2d21550eec71 None admin] RBAC: Adding query filter params (role.id=33c5141e078844bbb60127f2b26540d7, scope.project.id=009a328c01ce4fed9c5b90fec435b08f, user.id=6636e4878e5846e598b80dcba8afbeef) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:34:49.017782 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2336c5e1-ee0f-47e5-9ec7-2d21550eec71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:49.020376 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2336c5e1-ee0f-47e5-9ec7-2d21550eec71 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:49.029131 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 50/95] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:34:48 2022] GET /identity/v3/role_assignments?user.id=6636e4878e5846e598b80dcba8afbeef&scope.project.id=009a328c01ce4fed9c5b90fec435b08f&role.id=33c5141e078844bbb60127f2b26540d7 => generated 260 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:34:49.121200 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:49.133438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789f07c8-3624-46e3-a37d-504f2ea7b24c None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:49.195778 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b254d27-6e5f-4724-a87a-822bdd300e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:49.196678 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b254d27-6e5f-4724-a87a-822bdd300e29 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:49.197109 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b254d27-6e5f-4724-a87a-822bdd300e29 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:49.197544 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b254d27-6e5f-4724-a87a-822bdd300e29 None admin] PATH_INFO: `/v3/projects/459ff8f521284cf8b3ec858bfb1ba630` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:49.203984 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b254d27-6e5f-4724-a87a-822bdd300e29 None admin] RBAC: Authorizing `identity:get_project(project_id=459ff8f521284cf8b3ec858bfb1ba630)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:49.206424 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b254d27-6e5f-4724-a87a-822bdd300e29 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:49.209069 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 46/96] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:34:48 2022] GET /identity/v3/projects/459ff8f521284cf8b3ec858bfb1ba630 => generated 299 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:49.216431 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2336c5e1-ee0f-47e5-9ec7-2d21550eec71 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:49.231435 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:49.377208 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2336c5e1-ee0f-47e5-9ec7-2d21550eec71 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:49.434023 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ebf497-2eed-4067-872f-1909e427f3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:49.434023 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ebf497-2eed-4067-872f-1909e427f3a1 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:49.434023 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ebf497-2eed-4067-872f-1909e427f3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:49.434023 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ebf497-2eed-4067-872f-1909e427f3a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:49.434607 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ebf497-2eed-4067-872f-1909e427f3a1 None admin] RBAC: Adding query filter params (role.id=a4213d713b6f49d78d7bbb9484c9312f, scope.project.id=459ff8f521284cf8b3ec858bfb1ba630, user.id=6636e4878e5846e598b80dcba8afbeef) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:34:49.434831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ebf497-2eed-4067-872f-1909e427f3a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:49.436463 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ebf497-2eed-4067-872f-1909e427f3a1 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:49.450593 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 51/97] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:34:49 2022] GET /identity/v3/role_assignments?user.id=6636e4878e5846e598b80dcba8afbeef&scope.project.id=459ff8f521284cf8b3ec858bfb1ba630&role.id=a4213d713b6f49d78d7bbb9484c9312f => generated 260 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:50.846995 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064ab0d-855d-4317-a2b5-17c391c12f4c None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:50.847542 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064ab0d-855d-4317-a2b5-17c391c12f4c None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:50.847617 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064ab0d-855d-4317-a2b5-17c391c12f4c None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:50.849107 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 47/98] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:34:50 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:34:50.854879 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3ec5a-9607-42b9-b32e-feaf29d1e101 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:50.855408 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3ec5a-9607-42b9-b32e-feaf29d1e101 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:50.855690 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf3ec5a-9607-42b9-b32e-feaf29d1e101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:50.875777 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-fbf3ec5a-9607-42b9-b32e-feaf29d1e101 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:50.877017 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-fbf3ec5a-9607-42b9-b32e-feaf29d1e101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:50.996469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 52/99] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:34:50 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:34:51.032663 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064ab0d-855d-4317-a2b5-17c391c12f4c None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:51.187702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064ab0d-855d-4317-a2b5-17c391c12f4c None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:51.250438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e54cae-8660-4df6-8f68-6f244baf21e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:51.251177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e54cae-8660-4df6-8f68-6f244baf21e5 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:51.251458 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e54cae-8660-4df6-8f68-6f244baf21e5 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:51.251736 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e54cae-8660-4df6-8f68-6f244baf21e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:51.255959 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e54cae-8660-4df6-8f68-6f244baf21e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:51.257521 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e54cae-8660-4df6-8f68-6f244baf21e5 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:51.259814 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 48/100] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:34:51 2022] GET /identity/v3/domains/default => generated 200 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:51.267003 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3ec5a-9607-42b9-b32e-feaf29d1e101 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:51.269935 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:51.427452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf3ec5a-9607-42b9-b32e-feaf29d1e101 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:51.443097 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da95bfc-7172-4eda-ba31-e9a333f6e2cd None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:51.443336 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da95bfc-7172-4eda-ba31-e9a333f6e2cd None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:51.443648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da95bfc-7172-4eda-ba31-e9a333f6e2cd None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:51.445790 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 49/101] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:34:51 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:34:51.457352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af94a2-acec-4122-8965-dc28e25ffd8a None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:51.457352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af94a2-acec-4122-8965-dc28e25ffd8a None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:51.458117 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62af94a2-acec-4122-8965-dc28e25ffd8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:51.527817 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-62af94a2-acec-4122-8965-dc28e25ffd8a None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:51.529256 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-62af94a2-acec-4122-8965-dc28e25ffd8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:51.564283 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27962f0e-b63d-4de8-b3a6-14ee8feaa53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:51.565183 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27962f0e-b63d-4de8-b3a6-14ee8feaa53d None admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:51.565425 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27962f0e-b63d-4de8-b3a6-14ee8feaa53d None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:51.565763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27962f0e-b63d-4de8-b3a6-14ee8feaa53d None admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:51.566943 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27962f0e-b63d-4de8-b3a6-14ee8feaa53d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:51.568533 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27962f0e-b63d-4de8-b3a6-14ee8feaa53d None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:51.665820 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:51.676779 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 53/102] 10.222.0.115 () {60 vars in 1146 bytes} [Tue Aug 2 10:34:51 2022] POST /identity/v3/projects => generated 303 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:34:51.693888 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf257350-498c-45cb-ac5c-bb5b0614f7f4 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:51.694198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf257350-498c-45cb-ac5c-bb5b0614f7f4 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:51.694514 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf257350-498c-45cb-ac5c-bb5b0614f7f4 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:51.696774 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 54/103] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:34:51 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:34:51.725180 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d922e9a-0372-4cc4-911b-b38eda7d6a95 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:51.725560 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d922e9a-0372-4cc4-911b-b38eda7d6a95 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:51.725772 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d922e9a-0372-4cc4-911b-b38eda7d6a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:51.743283 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 50/104] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:34:51 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:34:51.772269 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-1d922e9a-0372-4cc4-911b-b38eda7d6a95 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:51.773574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-1d922e9a-0372-4cc4-911b-b38eda7d6a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:51.824817 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af94a2-acec-4122-8965-dc28e25ffd8a None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:51.869733 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:51.888091 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 55/105] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:34:51 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:34:51.929677 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d922e9a-0372-4cc4-911b-b38eda7d6a95 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:52.043937 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62af94a2-acec-4122-8965-dc28e25ffd8a None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:52.114720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744e42c-d16f-4ba8-b58a-245ba748954a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:52.115416 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744e42c-d16f-4ba8-b58a-245ba748954a None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:52.115759 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744e42c-d16f-4ba8-b58a-245ba748954a None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:52.116044 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744e42c-d16f-4ba8-b58a-245ba748954a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:52.130242 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1744e42c-d16f-4ba8-b58a-245ba748954a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:34:52.135767 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-1744e42c-d16f-4ba8-b58a-245ba748954a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:34:52.135767 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:34:52.135767 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:34:52.135767 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:34:52.135767 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:34:52.135767 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:34:52.135767 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:34:52.135767 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:34:52.135767 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:34:52.135767 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:34:52.135767 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:34:52.135767 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:34:52.135767 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:34:52.135767 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:34:52.135767 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:34:52.135767 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:34:52.136702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:34:52.136702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:34:52.136702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:34:52.136702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:34:52.136702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:34:52.136702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:34:52.136702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:34:52.136702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:34:52.136702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:34:52.136702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:34:52.136702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:34:52.136702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:34:52.136702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:34:52.136702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:34:52.136702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:34:52.136702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:34:52.137455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:34:52.137455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:34:52.137455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:34:52.137455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:34:52.137455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:34:52.137455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:34:52.137455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:34:52.137455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:34:52.137455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:34:52.137455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:34:52.137455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:34:52.137455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:34:52.137455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 51/106] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:34:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:34:52.143150 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744e42c-d16f-4ba8-b58a-245ba748954a None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:52.165599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d922e9a-0372-4cc4-911b-b38eda7d6a95 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:52.168496 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:52.243983 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b40c833-4d85-469f-8338-01e2e9f7c796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:52.244653 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40c833-4d85-469f-8338-01e2e9f7c796 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:52.246327 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40c833-4d85-469f-8338-01e2e9f7c796 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:52.246621 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b40c833-4d85-469f-8338-01e2e9f7c796 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:52.271695 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b40c833-4d85-469f-8338-01e2e9f7c796 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:34:52.277122 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-0b40c833-4d85-469f-8338-01e2e9f7c796 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:34:52.277122 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:34:52.277122 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:34:52.277122 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:34:52.277122 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:34:52.277122 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:34:52.277122 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:34:52.277122 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:34:52.277122 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:34:52.277122 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:34:52.277122 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:34:52.277122 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:34:52.277122 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:34:52.277122 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:34:52.277122 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:34:52.277122 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:34:52.278893 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:34:52.278893 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:34:52.278893 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:34:52.278893 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:34:52.278893 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:34:52.278893 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:34:52.278893 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:34:52.278893 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:34:52.278893 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:34:52.278893 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:34:52.278893 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:34:52.278893 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:34:52.278893 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:34:52.278893 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:34:52.278893 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:34:52.278893 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:34:52.280596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:34:52.280596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:34:52.280596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:34:52.280596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:34:52.280596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:34:52.280596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:34:52.280596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:34:52.280596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:34:52.280596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:34:52.280596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:34:52.280596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:34:52.280596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:34:52.280596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 56/107] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:34:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:34:52.280596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbda2da-d5e6-4a79-8ae1-41ef23ad1163 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:52.282621 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbda2da-d5e6-4a79-8ae1-41ef23ad1163 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:52.282621 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbda2da-d5e6-4a79-8ae1-41ef23ad1163 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:52.282621 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 57/108] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:34:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:34:52.285318 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea311cde-4a4e-4ad1-b38a-28828178a14f None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:52.285318 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea311cde-4a4e-4ad1-b38a-28828178a14f None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:52.285318 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea311cde-4a4e-4ad1-b38a-28828178a14f None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:52.291547 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 58/109] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:34:52 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:34:52.291547 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a2775-f376-437f-89d7-1b9227dacb24 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:52.291547 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a2775-f376-437f-89d7-1b9227dacb24 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:52.291547 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a2775-f376-437f-89d7-1b9227dacb24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:52.332823 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-2b4a2775-f376-437f-89d7-1b9227dacb24 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:52.332823 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-2b4a2775-f376-437f-89d7-1b9227dacb24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:52.333363 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744e42c-d16f-4ba8-b58a-245ba748954a None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:52.383898 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6426c9f-8f65-4fbf-be71-47e463a8964d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:52.384536 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6426c9f-8f65-4fbf-be71-47e463a8964d None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:52.384817 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6426c9f-8f65-4fbf-be71-47e463a8964d None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:52.385104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6426c9f-8f65-4fbf-be71-47e463a8964d None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:52.385823 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6426c9f-8f65-4fbf-be71-47e463a8964d None admin] RBAC: Adding query filter params (name=reader) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:34:52.386137 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6426c9f-8f65-4fbf-be71-47e463a8964d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:52.390761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6426c9f-8f65-4fbf-be71-47e463a8964d None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:52.402491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:52.403502 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 52/110] 10.222.0.115 () {56 vars in 1106 bytes} [Tue Aug 2 10:34:52 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:34:52.405633 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6426c9f-8f65-4fbf-be71-47e463a8964d None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:52.438481 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 59/111] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:34:52 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:34:52.440185 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6cd7c-95cc-4a3c-aaab-4a6ccc04ed0d None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:52.440481 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6cd7c-95cc-4a3c-aaab-4a6ccc04ed0d None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:52.447322 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add6cd7c-95cc-4a3c-aaab-4a6ccc04ed0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:52.467303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-add6cd7c-95cc-4a3c-aaab-4a6ccc04ed0d None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:52.468665 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-add6cd7c-95cc-4a3c-aaab-4a6ccc04ed0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:52.525565 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:52.555678 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6426c9f-8f65-4fbf-be71-47e463a8964d None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:52.560127 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 60/112] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:34:52 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:34:52.561185 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add6cd7c-95cc-4a3c-aaab-4a6ccc04ed0d None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:52.595849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759c605-0eb3-4d0b-8023-b457c6f39785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:52.597282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759c605-0eb3-4d0b-8023-b457c6f39785 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:52.597683 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759c605-0eb3-4d0b-8023-b457c6f39785 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:52.597966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759c605-0eb3-4d0b-8023-b457c6f39785 None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:52.598695 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6759c605-0eb3-4d0b-8023-b457c6f39785 None admin] RBAC: Adding query filter params (name=member) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:34:52.601943 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6759c605-0eb3-4d0b-8023-b457c6f39785 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:52.605924 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6759c605-0eb3-4d0b-8023-b457c6f39785 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:52.615432 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 53/113] 10.222.0.115 () {56 vars in 1106 bytes} [Tue Aug 2 10:34:52 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:52.616457 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759c605-0eb3-4d0b-8023-b457c6f39785 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:52.700538 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add6cd7c-95cc-4a3c-aaab-4a6ccc04ed0d None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:52.704455 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:52.726083 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759c605-0eb3-4d0b-8023-b457c6f39785 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:52.740420 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9f68d-b565-4753-9ff6-8d4c1ed5db7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:52.741395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9f68d-b565-4753-9ff6-8d4c1ed5db7e None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:52.741841 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9f68d-b565-4753-9ff6-8d4c1ed5db7e None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:52.762644 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a9f68d-b565-4753-9ff6-8d4c1ed5db7e None admin] PATH_INFO: `/v3/users/35dea9b8dc7d4cc2b2e739876cc41d55` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:52.762644 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a9f68d-b565-4753-9ff6-8d4c1ed5db7e None admin] RBAC: Authorizing `identity:get_user(user_id=35dea9b8dc7d4cc2b2e739876cc41d55)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:52.762644 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a9f68d-b565-4753-9ff6-8d4c1ed5db7e None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:52.762644 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 61/114] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:34:52 2022] GET /identity/v3/users/35dea9b8dc7d4cc2b2e739876cc41d55 => generated 290 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:52.762644 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9f68d-b565-4753-9ff6-8d4c1ed5db7e None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:52.764287 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27b5bf-503f-492b-9b08-ca5df8287c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:52.765072 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27b5bf-503f-492b-9b08-ca5df8287c3a None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:52.765422 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27b5bf-503f-492b-9b08-ca5df8287c3a None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:52.765774 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c27b5bf-503f-492b-9b08-ca5df8287c3a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:52.778660 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c27b5bf-503f-492b-9b08-ca5df8287c3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:34:52.786053 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-5c27b5bf-503f-492b-9b08-ca5df8287c3a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:34:52.786053 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:34:52.786053 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:34:52.786053 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:34:52.786053 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:34:52.786053 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:34:52.786053 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:34:52.786053 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:34:52.786053 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:34:52.786053 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:34:52.786053 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:34:52.786053 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:34:52.786053 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:34:52.786053 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:34:52.786053 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:34:52.786053 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:34:52.787036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:34:52.787036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:34:52.787036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:34:52.787036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:34:52.787036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:34:52.787036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:34:52.787036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:34:52.787036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:34:52.787036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:34:52.787036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:34:52.787036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:34:52.787036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:34:52.787036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:34:52.787036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:34:52.787036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:34:52.787036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:34:52.788184 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:34:52.788184 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:34:52.788184 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:34:52.788184 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:34:52.788184 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:34:52.788184 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:34:52.788184 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:34:52.788184 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:34:52.788184 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:34:52.788184 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:34:52.788184 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:34:52.788184 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:34:52.788184 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 54/115] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:34:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:34:52.792489 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27b5bf-503f-492b-9b08-ca5df8287c3a None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:52.807264 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:52.920181 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a9f68d-b565-4753-9ff6-8d4c1ed5db7e None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:52.925980 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c27b5bf-503f-492b-9b08-ca5df8287c3a None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:52.966266 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a578f30-a465-43dd-806c-05f88d9524a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:52.967429 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a578f30-a465-43dd-806c-05f88d9524a4 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:52.967821 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a578f30-a465-43dd-806c-05f88d9524a4 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:52.968467 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a578f30-a465-43dd-806c-05f88d9524a4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:52.973379 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9075b-8a18-459e-ae7a-688835b46753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:52.974284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9075b-8a18-459e-ae7a-688835b46753 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:52.974770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9075b-8a18-459e-ae7a-688835b46753 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:52.975328 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9075b-8a18-459e-ae7a-688835b46753 None admin] PATH_INFO: `/v3/users/6636e4878e5846e598b80dcba8afbeef` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:52.976737 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9075b-8a18-459e-ae7a-688835b46753 None admin] RBAC: Authorizing `identity:get_user(user_id=6636e4878e5846e598b80dcba8afbeef)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:52.978404 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9075b-8a18-459e-ae7a-688835b46753 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:52.980499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 55/116] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:34:52 2022] GET /identity/v3/users/6636e4878e5846e598b80dcba8afbeef => generated 276 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:52.982131 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a578f30-a465-43dd-806c-05f88d9524a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:34:52.982599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9075b-8a18-459e-ae7a-688835b46753 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:52.996606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-4a578f30-a465-43dd-806c-05f88d9524a4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:34:52.996606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:34:52.996606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:34:52.996606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:34:52.996606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:34:52.996606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:34:52.996606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:34:52.996606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:34:52.996606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:34:52.996606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:34:52.996606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:34:52.996606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:34:52.996606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:34:52.996606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:34:52.996606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:34:52.996606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:34:52.997649 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:34:52.997649 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:34:52.997649 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:34:52.997649 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:34:52.997649 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:34:52.997649 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:34:52.997649 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:34:52.997649 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:34:52.997649 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:34:52.997649 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:34:52.997649 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:34:52.997649 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:34:52.997649 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:34:52.997649 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:34:52.997649 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:34:52.997649 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:34:52.998438 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:34:52.998438 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:34:52.998438 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:34:52.998438 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:34:52.998438 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:34:52.998438 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:34:52.998438 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:34:52.998438 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:34:52.998438 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:34:52.998438 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:34:52.998438 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:34:52.998438 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:34:52.998438 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 62/117] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:34:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:34:52.998438 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a578f30-a465-43dd-806c-05f88d9524a4 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:53.021950 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:53.112764 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:53.125000 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a578f30-a465-43dd-806c-05f88d9524a4 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:53.138069 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9075b-8a18-459e-ae7a-688835b46753 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:53.170275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441927c1-b7f3-4b59-8561-e8c7d7f3a850 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:53.171327 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441927c1-b7f3-4b59-8561-e8c7d7f3a850 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:53.171808 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441927c1-b7f3-4b59-8561-e8c7d7f3a850 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:53.172280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441927c1-b7f3-4b59-8561-e8c7d7f3a850 None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:53.173448 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441927c1-b7f3-4b59-8561-e8c7d7f3a850 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:34:53.174522 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441927c1-b7f3-4b59-8561-e8c7d7f3a850 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:53.176772 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441927c1-b7f3-4b59-8561-e8c7d7f3a850 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:53.183208 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed85c6-ab3e-4064-b57a-2e325c884ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:53.185100 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed85c6-ab3e-4064-b57a-2e325c884ae1 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:53.185381 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed85c6-ab3e-4064-b57a-2e325c884ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:53.185653 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daed85c6-ab3e-4064-b57a-2e325c884ae1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:53.188798 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 63/118] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:34:52 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:53.191472 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441927c1-b7f3-4b59-8561-e8c7d7f3a850 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:53.276147 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed85c6-ab3e-4064-b57a-2e325c884ae1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:53.277881 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daed85c6-ab3e-4064-b57a-2e325c884ae1 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:53.307481 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:53.327541 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441927c1-b7f3-4b59-8561-e8c7d7f3a850 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:53.365047 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:53.373959 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4686946-7bfd-4d40-b786-a2765a9954bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:53.374946 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4686946-7bfd-4d40-b786-a2765a9954bd None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:53.375424 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4686946-7bfd-4d40-b786-a2765a9954bd None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:53.375887 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4686946-7bfd-4d40-b786-a2765a9954bd None admin] PATH_INFO: `/v3/projects/459ff8f521284cf8b3ec858bfb1ba630` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:53.378512 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4686946-7bfd-4d40-b786-a2765a9954bd None admin] RBAC: Authorizing `identity:get_project(project_id=459ff8f521284cf8b3ec858bfb1ba630)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:53.380020 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4686946-7bfd-4d40-b786-a2765a9954bd None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:53.381749 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 64/119] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:34:53 2022] GET /identity/v3/projects/459ff8f521284cf8b3ec858bfb1ba630 => generated 299 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:53.382593 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4686946-7bfd-4d40-b786-a2765a9954bd None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:53.415309 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 56/120] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:34:52 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:34:53.416598 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed85c6-ab3e-4064-b57a-2e325c884ae1 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:53.533221 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4686946-7bfd-4d40-b786-a2765a9954bd None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:53.571935 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b89acc2-6a8d-440a-96a8-210df04ced00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:53.572138 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daed85c6-ab3e-4064-b57a-2e325c884ae1 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:53.572940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89acc2-6a8d-440a-96a8-210df04ced00 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:53.573234 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89acc2-6a8d-440a-96a8-210df04ced00 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:53.573517 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89acc2-6a8d-440a-96a8-210df04ced00 None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:53.574354 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b89acc2-6a8d-440a-96a8-210df04ced00 None admin] RBAC: Adding query filter params (name=member) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:34:53.574662 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b89acc2-6a8d-440a-96a8-210df04ced00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:53.576101 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b89acc2-6a8d-440a-96a8-210df04ced00 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:53.588511 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 65/121] 10.222.0.115 () {56 vars in 1106 bytes} [Tue Aug 2 10:34:53 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:53.590012 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b5499-6fd3-4140-9188-f550bb6f308e None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:53.590433 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b5499-6fd3-4140-9188-f550bb6f308e None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:53.590655 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5b5499-6fd3-4140-9188-f550bb6f308e None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:53.592276 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 66/122] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:34:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:34:53.592743 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b5499-6fd3-4140-9188-f550bb6f308e None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:53.612279 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:53.623757 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5dc964-71e8-42e9-97d2-160e6275fafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:53.624693 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5dc964-71e8-42e9-97d2-160e6275fafe None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:53.625154 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5dc964-71e8-42e9-97d2-160e6275fafe None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:53.625603 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5dc964-71e8-42e9-97d2-160e6275fafe None admin] PATH_INFO: `/v3/users/4fb0c392df554f5dbd68f6861be954a1` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:53.626669 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5dc964-71e8-42e9-97d2-160e6275fafe None admin] RBAC: Authorizing `identity:get_user(user_id=4fb0c392df554f5dbd68f6861be954a1)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:53.633119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5dc964-71e8-42e9-97d2-160e6275fafe None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:53.633119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 57/123] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:34:53 2022] GET /identity/v3/users/4fb0c392df554f5dbd68f6861be954a1 => generated 248 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:53.633119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5dc964-71e8-42e9-97d2-160e6275fafe None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:53.676264 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:53.756474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5b5499-6fd3-4140-9188-f550bb6f308e None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:53.775120 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5dc964-71e8-42e9-97d2-160e6275fafe None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:53.806989 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5d75b2-eab0-49a1-a58d-127ad68307a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:53.807729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d75b2-eab0-49a1-a58d-127ad68307a8 None admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:53.807972 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d75b2-eab0-49a1-a58d-127ad68307a8 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:53.808251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d75b2-eab0-49a1-a58d-127ad68307a8 None admin] PATH_INFO: `/v3/projects/459ff8f521284cf8b3ec858bfb1ba630/users/6636e4878e5846e598b80dcba8afbeef/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:53.810203 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5d75b2-eab0-49a1-a58d-127ad68307a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=459ff8f521284cf8b3ec858bfb1ba630, user_id=6636e4878e5846e598b80dcba8afbeef, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:53.811738 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5d75b2-eab0-49a1-a58d-127ad68307a8 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:53.817201 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59f26e6-45ee-408c-8d85-68dab4020ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:53.818080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f26e6-45ee-408c-8d85-68dab4020ab2 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:53.818414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f26e6-45ee-408c-8d85-68dab4020ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:53.818838 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f26e6-45ee-408c-8d85-68dab4020ab2 None admin] PATH_INFO: `/v3/projects/459ff8f521284cf8b3ec858bfb1ba630` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:53.820243 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59f26e6-45ee-408c-8d85-68dab4020ab2 None admin] RBAC: Authorizing `identity:get_project(project_id=459ff8f521284cf8b3ec858bfb1ba630)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:53.822019 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59f26e6-45ee-408c-8d85-68dab4020ab2 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:53.823951 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 58/124] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:34:53 2022] GET /identity/v3/projects/459ff8f521284cf8b3ec858bfb1ba630 => generated 299 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:53.830690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea7414-5a19-4d0c-8a61-2a2e66d369e0 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:53.830690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea7414-5a19-4d0c-8a61-2a2e66d369e0 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:53.830690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea7414-5a19-4d0c-8a61-2a2e66d369e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:53.848175 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-6dea7414-5a19-4d0c-8a61-2a2e66d369e0 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:53.850184 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-6dea7414-5a19-4d0c-8a61-2a2e66d369e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:53.934410 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 67/125] 10.222.0.115 () {58 vars in 1444 bytes} [Tue Aug 2 10:34:53 2022] PUT /identity/v3/projects/459ff8f521284cf8b3ec858bfb1ba630/users/6636e4878e5846e598b80dcba8afbeef/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:34:53.937053 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5d75b2-eab0-49a1-a58d-127ad68307a8 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:53.941828 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 59/126] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:34:53 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:34:53.944098 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea7414-5a19-4d0c-8a61-2a2e66d369e0 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:53.998478 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:54.067266 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:54.094412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5d75b2-eab0-49a1-a58d-127ad68307a8 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:54.100887 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea7414-5a19-4d0c-8a61-2a2e66d369e0 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:54.150860 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634510f-18bc-4543-b4c1-e807d9462541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:54.152104 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634510f-18bc-4543-b4c1-e807d9462541 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:54.152385 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634510f-18bc-4543-b4c1-e807d9462541 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:54.152662 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634510f-18bc-4543-b4c1-e807d9462541 None admin] PATH_INFO: `/v3/users/6636e4878e5846e598b80dcba8afbeef` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:54.154219 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634510f-18bc-4543-b4c1-e807d9462541 None admin] RBAC: Authorizing `identity:get_user(user_id=6636e4878e5846e598b80dcba8afbeef)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:54.156045 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634510f-18bc-4543-b4c1-e807d9462541 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:54.158414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 68/127] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:34:53 2022] GET /identity/v3/users/6636e4878e5846e598b80dcba8afbeef => generated 276 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:54.159440 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634510f-18bc-4543-b4c1-e807d9462541 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:54.183804 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b445e2-b70b-4724-b347-ed56f64fd0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:54.184482 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b445e2-b70b-4724-b347-ed56f64fd0ef None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:54.184756 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b445e2-b70b-4724-b347-ed56f64fd0ef None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:54.185042 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b445e2-b70b-4724-b347-ed56f64fd0ef None admin] PATH_INFO: `/v3/projects/459ff8f521284cf8b3ec858bfb1ba630` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:54.186652 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b445e2-b70b-4724-b347-ed56f64fd0ef None admin] RBAC: Authorizing `identity:get_project(project_id=459ff8f521284cf8b3ec858bfb1ba630)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:54.188823 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b445e2-b70b-4724-b347-ed56f64fd0ef None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:54.192153 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 60/128] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:34:53 2022] GET /identity/v3/projects/459ff8f521284cf8b3ec858bfb1ba630 => generated 299 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:54.193220 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b445e2-b70b-4724-b347-ed56f64fd0ef None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:54.300386 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634510f-18bc-4543-b4c1-e807d9462541 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:54.309171 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b445e2-b70b-4724-b347-ed56f64fd0ef None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:54.310891 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:54.343243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa8770-0444-4743-ae92-5bf8ee83edff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:54.344150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa8770-0444-4743-ae92-5bf8ee83edff None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:54.344613 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa8770-0444-4743-ae92-5bf8ee83edff None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:54.344919 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaa8770-0444-4743-ae92-5bf8ee83edff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:54.346523 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaa8770-0444-4743-ae92-5bf8ee83edff None admin] RBAC: Adding query filter params (role.id=1cf433b432764fbab711f54dcf7aa81e, scope.project.id=459ff8f521284cf8b3ec858bfb1ba630, user.id=35dea9b8dc7d4cc2b2e739876cc41d55) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:34:54.346886 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaa8770-0444-4743-ae92-5bf8ee83edff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:54.348451 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaa8770-0444-4743-ae92-5bf8ee83edff None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:54.352524 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2de30-0cd7-44a9-bb68-65d8793a9d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:54.353172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2de30-0cd7-44a9-bb68-65d8793a9d4b None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:54.353457 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2de30-0cd7-44a9-bb68-65d8793a9d4b None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:54.353735 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d2de30-0cd7-44a9-bb68-65d8793a9d4b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:54.356195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 69/129] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:34:54 2022] GET /identity/v3/role_assignments?user.id=35dea9b8dc7d4cc2b2e739876cc41d55&scope.project.id=459ff8f521284cf8b3ec858bfb1ba630&role.id=1cf433b432764fbab711f54dcf7aa81e => generated 260 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:54.357269 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa8770-0444-4743-ae92-5bf8ee83edff None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:54.375688 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:54.383432 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d2de30-0cd7-44a9-bb68-65d8793a9d4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:34:54.389518 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-d8d2de30-0cd7-44a9-bb68-65d8793a9d4b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:34:54.389518 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:34:54.389518 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:34:54.389518 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:34:54.389518 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:34:54.389518 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:34:54.389518 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:34:54.389518 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:34:54.389518 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:34:54.389518 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:34:54.389518 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:34:54.389518 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:34:54.389518 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:34:54.389518 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:34:54.389518 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:34:54.389518 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:34:54.402609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:34:54.402609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:34:54.402609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:34:54.402609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:34:54.402609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:34:54.402609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:34:54.402609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:34:54.402609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:34:54.402609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:34:54.402609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:34:54.402609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:34:54.402609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:34:54.402609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:34:54.402609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:34:54.402609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:34:54.402609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:34:54.403444 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:34:54.403444 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:34:54.403444 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:34:54.403444 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:34:54.403444 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:34:54.403444 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:34:54.403444 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:34:54.403444 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:34:54.403444 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:34:54.403444 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:34:54.403444 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:34:54.403444 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:34:54.403444 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 61/130] 10.222.0.115 () {56 vars in 1119 bytes} [Tue Aug 2 10:34:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:34:54.403444 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2de30-0cd7-44a9-bb68-65d8793a9d4b None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:54.528283 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaa8770-0444-4743-ae92-5bf8ee83edff None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:54.572909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5a526-835c-4a35-9911-31b66e2aae1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:54.573580 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5a526-835c-4a35-9911-31b66e2aae1b None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:54.573867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5a526-835c-4a35-9911-31b66e2aae1b None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:54.574148 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5a526-835c-4a35-9911-31b66e2aae1b None admin] PATH_INFO: `/v3/projects/009a328c01ce4fed9c5b90fec435b08f` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:54.575478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc5a526-835c-4a35-9911-31b66e2aae1b None admin] RBAC: Authorizing `identity:get_project(project_id=009a328c01ce4fed9c5b90fec435b08f)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:54.575951 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d2de30-0cd7-44a9-bb68-65d8793a9d4b None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:54.576960 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc5a526-835c-4a35-9911-31b66e2aae1b None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:54.580934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 70/131] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:34:54 2022] GET /identity/v3/projects/009a328c01ce4fed9c5b90fec435b08f => generated 313 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:54.582904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9175c-9554-4e90-be26-5f8ea7d7aae4 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:54.583426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9175c-9554-4e90-be26-5f8ea7d7aae4 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:54.583885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c9175c-9554-4e90-be26-5f8ea7d7aae4 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:54.590963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 71/132] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:34:54 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:34:54.590963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9175c-9554-4e90-be26-5f8ea7d7aae4 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:54.638730 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5519a9b-2928-4ff5-9db2-4d57152b990e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:54.640126 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5519a9b-2928-4ff5-9db2-4d57152b990e None admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:54.640850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5519a9b-2928-4ff5-9db2-4d57152b990e None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:54.641486 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5519a9b-2928-4ff5-9db2-4d57152b990e None admin] PATH_INFO: `/v3/projects/459ff8f521284cf8b3ec858bfb1ba630/users/4fb0c392df554f5dbd68f6861be954a1/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:54.645641 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5519a9b-2928-4ff5-9db2-4d57152b990e None admin] RBAC: Authorizing `identity:create_grant(project_id=459ff8f521284cf8b3ec858bfb1ba630, user_id=4fb0c392df554f5dbd68f6861be954a1, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:54.648303 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5519a9b-2928-4ff5-9db2-4d57152b990e None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:54.703922 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:54.736983 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c9175c-9554-4e90-be26-5f8ea7d7aae4 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:54.777049 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ceb796-9615-467f-902b-63cb46cf8bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:54.777704 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ceb796-9615-467f-902b-63cb46cf8bc4 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:54.777965 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ceb796-9615-467f-902b-63cb46cf8bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:54.778243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ceb796-9615-467f-902b-63cb46cf8bc4 None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:54.778976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ceb796-9615-467f-902b-63cb46cf8bc4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:34:54.779312 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ceb796-9615-467f-902b-63cb46cf8bc4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:54.780917 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ceb796-9615-467f-902b-63cb46cf8bc4 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:54.795910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 72/133] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:34:54 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:54.795910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ceb796-9615-467f-902b-63cb46cf8bc4 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:54.810617 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 62/134] 10.222.0.115 () {58 vars in 1444 bytes} [Tue Aug 2 10:34:54 2022] PUT /identity/v3/projects/459ff8f521284cf8b3ec858bfb1ba630/users/4fb0c392df554f5dbd68f6861be954a1/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:34:54.817207 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa0b9e-956a-445a-a178-b0e8c71bd7c9 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:54.818025 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa0b9e-956a-445a-a178-b0e8c71bd7c9 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:54.822201 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa0b9e-956a-445a-a178-b0e8c71bd7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:54.864210 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-87fa0b9e-956a-445a-a178-b0e8c71bd7c9 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:54.865573 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-87fa0b9e-956a-445a-a178-b0e8c71bd7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:54.899356 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:54.945183 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ceb796-9615-467f-902b-63cb46cf8bc4 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:55.014351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 63/135] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:34:54 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:34:55.019369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa0b9e-956a-445a-a178-b0e8c71bd7c9 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:55.038909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05f0f5-bf69-4b42-a8c7-e18706bfa133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:55.039631 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05f0f5-bf69-4b42-a8c7-e18706bfa133 None admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:55.039910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05f0f5-bf69-4b42-a8c7-e18706bfa133 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:55.040235 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c05f0f5-bf69-4b42-a8c7-e18706bfa133 None admin] PATH_INFO: `/v3/projects/009a328c01ce4fed9c5b90fec435b08f/users/6636e4878e5846e598b80dcba8afbeef/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:55.045117 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05f0f5-bf69-4b42-a8c7-e18706bfa133 None admin] RBAC: Authorizing `identity:create_grant(project_id=009a328c01ce4fed9c5b90fec435b08f, user_id=6636e4878e5846e598b80dcba8afbeef, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:55.046585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c05f0f5-bf69-4b42-a8c7-e18706bfa133 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:55.050749 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:55.151088 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa0b9e-956a-445a-a178-b0e8c71bd7c9 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:55.166437 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 73/136] 10.222.0.115 () {58 vars in 1444 bytes} [Tue Aug 2 10:34:54 2022] PUT /identity/v3/projects/009a328c01ce4fed9c5b90fec435b08f/users/6636e4878e5846e598b80dcba8afbeef/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:34:55.168539 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05f0f5-bf69-4b42-a8c7-e18706bfa133 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:55.227118 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d68608-a103-4689-a92e-9183e0acca0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:55.228065 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d68608-a103-4689-a92e-9183e0acca0a None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:55.228355 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d68608-a103-4689-a92e-9183e0acca0a None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:55.228647 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d68608-a103-4689-a92e-9183e0acca0a None admin] PATH_INFO: `/v3/users/6636e4878e5846e598b80dcba8afbeef` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:55.230979 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d68608-a103-4689-a92e-9183e0acca0a None admin] RBAC: Authorizing `identity:get_user(user_id=6636e4878e5846e598b80dcba8afbeef)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:55.232539 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d68608-a103-4689-a92e-9183e0acca0a None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:55.234349 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 64/137] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:34:55 2022] GET /identity/v3/users/6636e4878e5846e598b80dcba8afbeef => generated 276 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:55.242447 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d68608-a103-4689-a92e-9183e0acca0a None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:55.301905 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:55.417996 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c05f0f5-bf69-4b42-a8c7-e18706bfa133 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:55.501893 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d68608-a103-4689-a92e-9183e0acca0a None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:55.525673 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0422a-127a-45e8-b9d8-b336bd81d3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:55.526756 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0422a-127a-45e8-b9d8-b336bd81d3e4 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:55.527303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0422a-127a-45e8-b9d8-b336bd81d3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:55.527792 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f0422a-127a-45e8-b9d8-b336bd81d3e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:55.531320 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f0422a-127a-45e8-b9d8-b336bd81d3e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:55.533093 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f0422a-127a-45e8-b9d8-b336bd81d3e4 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:55.536531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 74/138] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:34:55 2022] GET /identity/v3/domains/default => generated 200 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:55.542038 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0422a-127a-45e8-b9d8-b336bd81d3e4 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:55.561783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8ae70-4a42-43ea-8240-3339673fc5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:55.562426 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8ae70-4a42-43ea-8240-3339673fc5db None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:55.562703 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8ae70-4a42-43ea-8240-3339673fc5db None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:55.562977 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8ae70-4a42-43ea-8240-3339673fc5db None admin] PATH_INFO: `/v3/projects/459ff8f521284cf8b3ec858bfb1ba630` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:55.567636 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8ae70-4a42-43ea-8240-3339673fc5db None admin] RBAC: Authorizing `identity:get_project(project_id=459ff8f521284cf8b3ec858bfb1ba630)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:55.567636 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b8ae70-4a42-43ea-8240-3339673fc5db None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:55.567871 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 65/139] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:34:55 2022] GET /identity/v3/projects/459ff8f521284cf8b3ec858bfb1ba630 => generated 299 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:55.573884 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8ae70-4a42-43ea-8240-3339673fc5db None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:55.681517 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:55.698497 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8ae70-4a42-43ea-8240-3339673fc5db None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:55.710995 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:55.737704 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c3d36-1d35-4936-a247-f30f37cb0bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:55.738395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c3d36-1d35-4936-a247-f30f37cb0bf4 None admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:55.738680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c3d36-1d35-4936-a247-f30f37cb0bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:55.738952 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c3d36-1d35-4936-a247-f30f37cb0bf4 None admin] PATH_INFO: `/v3/projects/459ff8f521284cf8b3ec858bfb1ba630/users/6636e4878e5846e598b80dcba8afbeef/roles/a4213d713b6f49d78d7bbb9484c9312f` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:55.746639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550c3d36-1d35-4936-a247-f30f37cb0bf4 None admin] RBAC: Authorizing `identity:create_grant(project_id=459ff8f521284cf8b3ec858bfb1ba630, user_id=6636e4878e5846e598b80dcba8afbeef, role_id=a4213d713b6f49d78d7bbb9484c9312f)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:55.746639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550c3d36-1d35-4936-a247-f30f37cb0bf4 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:55.775740 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f0422a-127a-45e8-b9d8-b336bd81d3e4 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:55.835708 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 66/140] 10.222.0.115 () {58 vars in 1444 bytes} [Tue Aug 2 10:34:55 2022] PUT /identity/v3/projects/459ff8f521284cf8b3ec858bfb1ba630/users/6636e4878e5846e598b80dcba8afbeef/roles/a4213d713b6f49d78d7bbb9484c9312f => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:34:55.840362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12a1ee-aa3e-40fb-82fe-f47182521184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:55.841979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12a1ee-aa3e-40fb-82fe-f47182521184 None admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:55.842486 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12a1ee-aa3e-40fb-82fe-f47182521184 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:55.842980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12a1ee-aa3e-40fb-82fe-f47182521184 None admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:55.844336 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a12a1ee-aa3e-40fb-82fe-f47182521184 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:55.847084 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a12a1ee-aa3e-40fb-82fe-f47182521184 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:56.052290 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 75/141] 10.222.0.115 () {60 vars in 1137 bytes} [Tue Aug 2 10:34:55 2022] POST /identity/v3/users => generated 284 bytes in 511 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:34:56.921224 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce8d1f-d23f-4610-9262-4a5d922fc6ea None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:56.921804 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce8d1f-d23f-4610-9262-4a5d922fc6ea None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:56.921804 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ce8d1f-d23f-4610-9262-4a5d922fc6ea None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:56.923148 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 67/142] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:34:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:34:56.929097 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b9723-e6c3-46c4-a082-f0b1b9bf833e None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:56.929379 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b9723-e6c3-46c4-a082-f0b1b9bf833e None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:56.929657 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01b9723-e6c3-46c4-a082-f0b1b9bf833e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:56.952172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-f01b9723-e6c3-46c4-a082-f0b1b9bf833e None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:56.953392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-f01b9723-e6c3-46c4-a082-f0b1b9bf833e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:57.034368 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 76/143] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:34:56 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:34:57.048060 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82340e28-c856-47b1-a8f8-afcfa45a83d0 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:57.048363 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82340e28-c856-47b1-a8f8-afcfa45a83d0 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:57.049149 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82340e28-c856-47b1-a8f8-afcfa45a83d0 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:57.050956 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 68/144] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:34:57 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:34:57.056876 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff861e6-cc19-4910-b7fc-e09c708bab0d None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:57.057365 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff861e6-cc19-4910-b7fc-e09c708bab0d None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:57.057438 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff861e6-cc19-4910-b7fc-e09c708bab0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:57.081439 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-fff861e6-cc19-4910-b7fc-e09c708bab0d None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:57.082895 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-fff861e6-cc19-4910-b7fc-e09c708bab0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:57.098888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82340e28-c856-47b1-a8f8-afcfa45a83d0 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:57.167451 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:57.185301 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 77/145] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:34:57 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:34:57.237995 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff861e6-cc19-4910-b7fc-e09c708bab0d None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:57.244330 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82340e28-c856-47b1-a8f8-afcfa45a83d0 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:57.350035 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64109dc-c110-4b2d-87f4-58165c551a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:57.350695 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64109dc-c110-4b2d-87f4-58165c551a61 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:57.350978 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64109dc-c110-4b2d-87f4-58165c551a61 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:57.351274 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64109dc-c110-4b2d-87f4-58165c551a61 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:57.379726 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64109dc-c110-4b2d-87f4-58165c551a61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:34:57.389604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-b64109dc-c110-4b2d-87f4-58165c551a61 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:34:57.389604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:34:57.389604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:34:57.389604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:34:57.389604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:34:57.389604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:34:57.389604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:34:57.389604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:34:57.389604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:34:57.389604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:34:57.389604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:34:57.389604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:34:57.389604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:34:57.389604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:34:57.389604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:34:57.389604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:34:57.391020 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:34:57.391020 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:34:57.391020 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:34:57.391020 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:34:57.391020 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:34:57.391020 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:34:57.391020 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:34:57.391020 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:34:57.391020 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:34:57.391020 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:34:57.391020 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:34:57.391020 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:34:57.391020 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:34:57.391020 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:34:57.391020 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:34:57.391020 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:34:57.392146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:34:57.392146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:34:57.392146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:34:57.392146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:34:57.392146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:34:57.392146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:34:57.392146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:34:57.392146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:34:57.392146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:34:57.392146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:34:57.392146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:34:57.392146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:34:57.392146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 69/146] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:34:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:34:57.398765 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64109dc-c110-4b2d-87f4-58165c551a61 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:57.414161 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:57.434291 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff861e6-cc19-4910-b7fc-e09c708bab0d None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:57.478752 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486af6e-aa29-4f8d-82c2-714e764a2f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:57.479452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486af6e-aa29-4f8d-82c2-714e764a2f1c None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:57.479731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486af6e-aa29-4f8d-82c2-714e764a2f1c None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:57.480004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3486af6e-aa29-4f8d-82c2-714e764a2f1c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:57.492507 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3486af6e-aa29-4f8d-82c2-714e764a2f1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:34:57.498799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-3486af6e-aa29-4f8d-82c2-714e764a2f1c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:34:57.498799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:34:57.498799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:34:57.498799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:34:57.498799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:34:57.498799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:34:57.498799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:34:57.498799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:34:57.498799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:34:57.498799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:34:57.498799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:34:57.498799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:34:57.498799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:34:57.498799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:34:57.498799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:34:57.498799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:34:57.504884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:34:57.504884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:34:57.504884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:34:57.504884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:34:57.504884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:34:57.504884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:34:57.504884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:34:57.504884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:34:57.504884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:34:57.504884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:34:57.504884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:34:57.504884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:34:57.504884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:34:57.504884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:34:57.504884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:34:57.504884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:34:57.505773 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:34:57.505773 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:34:57.505773 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:34:57.505773 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:34:57.505773 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:34:57.505773 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:34:57.505773 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:34:57.505773 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:34:57.505773 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:34:57.505773 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:34:57.505773 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:34:57.505773 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:34:57.505773 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 78/147] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:34:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:34:57.510846 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486af6e-aa29-4f8d-82c2-714e764a2f1c None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:57.580404 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:57.613053 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64109dc-c110-4b2d-87f4-58165c551a61 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:57.677276 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf8087e-90fe-4d69-9cab-a0cba92b7b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:57.677904 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8087e-90fe-4d69-9cab-a0cba92b7b03 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:57.678180 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8087e-90fe-4d69-9cab-a0cba92b7b03 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:57.678460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf8087e-90fe-4d69-9cab-a0cba92b7b03 None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:57.679229 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf8087e-90fe-4d69-9cab-a0cba92b7b03 None admin] RBAC: Adding query filter params (name=member) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:34:57.679536 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf8087e-90fe-4d69-9cab-a0cba92b7b03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:57.680962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf8087e-90fe-4d69-9cab-a0cba92b7b03 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:57.681433 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3486af6e-aa29-4f8d-82c2-714e764a2f1c None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:57.694692 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 70/148] 10.222.0.115 () {56 vars in 1106 bytes} [Tue Aug 2 10:34:57 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:34:57.704097 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf8087e-90fe-4d69-9cab-a0cba92b7b03 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:57.739966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888fd8d8-48ff-4ef5-ad1d-fea9e511056c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:57.740499 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888fd8d8-48ff-4ef5-ad1d-fea9e511056c None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:57.740849 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888fd8d8-48ff-4ef5-ad1d-fea9e511056c None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:57.741123 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888fd8d8-48ff-4ef5-ad1d-fea9e511056c None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:57.741853 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888fd8d8-48ff-4ef5-ad1d-fea9e511056c None admin] RBAC: Adding query filter params (name=reader) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:34:57.742154 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888fd8d8-48ff-4ef5-ad1d-fea9e511056c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:57.745520 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888fd8d8-48ff-4ef5-ad1d-fea9e511056c None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:57.755228 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 79/149] 10.222.0.115 () {56 vars in 1106 bytes} [Tue Aug 2 10:34:57 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:57.761330 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888fd8d8-48ff-4ef5-ad1d-fea9e511056c None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:57.811958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:57.866469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf8087e-90fe-4d69-9cab-a0cba92b7b03 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:57.920888 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888fd8d8-48ff-4ef5-ad1d-fea9e511056c None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:57.923013 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f8a82-6642-4c39-ae5e-4d3b800412c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:57.924332 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f8a82-6642-4c39-ae5e-4d3b800412c9 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:57.924617 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f8a82-6642-4c39-ae5e-4d3b800412c9 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:57.924890 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921f8a82-6642-4c39-ae5e-4d3b800412c9 None admin] PATH_INFO: `/v3/users/6636e4878e5846e598b80dcba8afbeef` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:57.925175 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:57.927822 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f8a82-6642-4c39-ae5e-4d3b800412c9 None admin] RBAC: Authorizing `identity:get_user(user_id=6636e4878e5846e598b80dcba8afbeef)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:57.929308 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921f8a82-6642-4c39-ae5e-4d3b800412c9 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:57.931081 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 71/150] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:34:57 2022] GET /identity/v3/users/6636e4878e5846e598b80dcba8afbeef => generated 276 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:57.941028 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f8a82-6642-4c39-ae5e-4d3b800412c9 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:58.004986 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d3a46a-bd44-4557-b5e1-f8973df8191a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:58.005586 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3a46a-bd44-4557-b5e1-f8973df8191a None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:58.005836 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3a46a-bd44-4557-b5e1-f8973df8191a None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:58.006115 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d3a46a-bd44-4557-b5e1-f8973df8191a None admin] PATH_INFO: `/v3/users/35dea9b8dc7d4cc2b2e739876cc41d55` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:58.008876 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d3a46a-bd44-4557-b5e1-f8973df8191a None admin] RBAC: Authorizing `identity:get_user(user_id=35dea9b8dc7d4cc2b2e739876cc41d55)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:58.010385 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d3a46a-bd44-4557-b5e1-f8973df8191a None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:58.017369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 80/151] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:34:57 2022] GET /identity/v3/users/35dea9b8dc7d4cc2b2e739876cc41d55 => generated 290 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:58.023738 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d3a46a-bd44-4557-b5e1-f8973df8191a None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:58.104838 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921f8a82-6642-4c39-ae5e-4d3b800412c9 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:58.147634 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d3a46a-bd44-4557-b5e1-f8973df8191a None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:58.153644 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b443d95b-8137-4d05-8b43-76bba569dc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:58.154282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443d95b-8137-4d05-8b43-76bba569dc58 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:58.154625 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443d95b-8137-4d05-8b43-76bba569dc58 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:58.154909 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b443d95b-8137-4d05-8b43-76bba569dc58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:58.168187 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:58.184783 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b4c2b-a555-473b-8927-5a8c3b9a708a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:58.185431 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b4c2b-a555-473b-8927-5a8c3b9a708a None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:58.185720 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b4c2b-a555-473b-8927-5a8c3b9a708a None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:58.185972 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b4c2b-a555-473b-8927-5a8c3b9a708a None admin] PATH_INFO: `/v3/projects/459ff8f521284cf8b3ec858bfb1ba630` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:58.187372 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674b4c2b-a555-473b-8927-5a8c3b9a708a None admin] RBAC: Authorizing `identity:get_project(project_id=459ff8f521284cf8b3ec858bfb1ba630)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:58.188829 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674b4c2b-a555-473b-8927-5a8c3b9a708a None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:58.191502 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 81/152] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:34:58 2022] GET /identity/v3/projects/459ff8f521284cf8b3ec858bfb1ba630 => generated 299 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:34:58.192726 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5965c-f5eb-43ef-8e7d-333ec2d10b38 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:58.193114 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5965c-f5eb-43ef-8e7d-333ec2d10b38 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:58.193302 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f5965c-f5eb-43ef-8e7d-333ec2d10b38 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:58.194543 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 82/153] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:34:58 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:34:58.200044 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb118878-1a1d-4bbc-9a6a-76c0a770ca34 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:58.200332 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb118878-1a1d-4bbc-9a6a-76c0a770ca34 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:58.200610 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb118878-1a1d-4bbc-9a6a-76c0a770ca34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:58.223016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-eb118878-1a1d-4bbc-9a6a-76c0a770ca34 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:58.224266 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-eb118878-1a1d-4bbc-9a6a-76c0a770ca34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:58.270972 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:58.271365 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b443d95b-8137-4d05-8b43-76bba569dc58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:58.272877 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b443d95b-8137-4d05-8b43-76bba569dc58 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:58.299756 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 83/154] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:34:58 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:34:58.308018 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb118878-1a1d-4bbc-9a6a-76c0a770ca34 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:58.434133 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb118878-1a1d-4bbc-9a6a-76c0a770ca34 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:58.447780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 72/155] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:34:57 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:34:58.449029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b443d95b-8137-4d05-8b43-76bba569dc58 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:58.480256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7d640-895b-428b-be7a-24439d6c9985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:58.480999 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7d640-895b-428b-be7a-24439d6c9985 None admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:58.481283 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7d640-895b-428b-be7a-24439d6c9985 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:58.481557 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b7d640-895b-428b-be7a-24439d6c9985 None admin] PATH_INFO: `/v3/projects/459ff8f521284cf8b3ec858bfb1ba630/users/35dea9b8dc7d4cc2b2e739876cc41d55/roles/1cf433b432764fbab711f54dcf7aa81e` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:58.484341 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b7d640-895b-428b-be7a-24439d6c9985 None admin] RBAC: Authorizing `identity:create_grant(project_id=459ff8f521284cf8b3ec858bfb1ba630, user_id=35dea9b8dc7d4cc2b2e739876cc41d55, role_id=1cf433b432764fbab711f54dcf7aa81e)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:58.485794 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b7d640-895b-428b-be7a-24439d6c9985 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:58.524394 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:58.580783 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 84/156] 10.222.0.115 () {58 vars in 1444 bytes} [Tue Aug 2 10:34:58 2022] PUT /identity/v3/projects/459ff8f521284cf8b3ec858bfb1ba630/users/35dea9b8dc7d4cc2b2e739876cc41d55/roles/1cf433b432764fbab711f54dcf7aa81e => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:34:58.583020 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7d640-895b-428b-be7a-24439d6c9985 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:58.608988 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b443d95b-8137-4d05-8b43-76bba569dc58 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:58.644480 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:58.672058 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5c370-0d82-4eae-8b7d-3090088aced9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:58.673415 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5c370-0d82-4eae-8b7d-3090088aced9 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:58.673927 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5c370-0d82-4eae-8b7d-3090088aced9 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:58.674406 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f5c370-0d82-4eae-8b7d-3090088aced9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:58.701666 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f5c370-0d82-4eae-8b7d-3090088aced9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:34:58.708364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-64f5c370-0d82-4eae-8b7d-3090088aced9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:34:58.708364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:34:58.708364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:34:58.708364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:34:58.708364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:34:58.708364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:34:58.708364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:34:58.708364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:34:58.708364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:34:58.708364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:34:58.708364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:34:58.708364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:34:58.708364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:34:58.708364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:34:58.708364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:34:58.708364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:34:58.709311 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:34:58.709311 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:34:58.709311 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:34:58.709311 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:34:58.709311 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:34:58.709311 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:34:58.709311 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:34:58.709311 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:34:58.709311 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:34:58.709311 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:34:58.709311 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:34:58.709311 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:34:58.709311 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:34:58.709311 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:34:58.709311 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:34:58.709311 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:34:58.710106 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:34:58.710106 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:34:58.710106 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:34:58.710106 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:34:58.710106 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:34:58.710106 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:34:58.710106 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:34:58.710106 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:34:58.710106 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:34:58.710106 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:34:58.710106 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:34:58.710106 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:34:58.713600 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 73/157] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:34:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:34:58.719971 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5c370-0d82-4eae-8b7d-3090088aced9 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:58.781729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b7d640-895b-428b-be7a-24439d6c9985 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:58.857705 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864bc51-488d-4de9-8e6b-f531777be09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:58.858457 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864bc51-488d-4de9-8e6b-f531777be09c None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:58.858735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864bc51-488d-4de9-8e6b-f531777be09c None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:58.859006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d864bc51-488d-4de9-8e6b-f531777be09c None admin] PATH_INFO: `/v3/projects/459ff8f521284cf8b3ec858bfb1ba630` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:58.864016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d864bc51-488d-4de9-8e6b-f531777be09c None admin] RBAC: Authorizing `identity:get_project(project_id=459ff8f521284cf8b3ec858bfb1ba630)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:58.865512 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d864bc51-488d-4de9-8e6b-f531777be09c None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:58.867924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 85/158] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:34:58 2022] GET /identity/v3/projects/459ff8f521284cf8b3ec858bfb1ba630 => generated 299 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:58.873860 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864bc51-488d-4de9-8e6b-f531777be09c None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:58.894180 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:58.903266 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f5c370-0d82-4eae-8b7d-3090088aced9 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:58.961271 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8897a-6691-45ae-821f-09ad1dc454ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:58.961908 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8897a-6691-45ae-821f-09ad1dc454ba None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:58.962212 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8897a-6691-45ae-821f-09ad1dc454ba None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:58.962457 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d8897a-6691-45ae-821f-09ad1dc454ba None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:58.963227 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8897a-6691-45ae-821f-09ad1dc454ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:34:58.963539 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8897a-6691-45ae-821f-09ad1dc454ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:58.965003 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d8897a-6691-45ae-821f-09ad1dc454ba None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:58.979230 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 74/159] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:34:58 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:58.994860 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8897a-6691-45ae-821f-09ad1dc454ba None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:59.079665 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d864bc51-488d-4de9-8e6b-f531777be09c None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:59.096281 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:59.128039 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623886a8-1267-4c67-9732-c75c9a2b2a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:59.128680 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623886a8-1267-4c67-9732-c75c9a2b2a94 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:59.128964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623886a8-1267-4c67-9732-c75c9a2b2a94 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:59.129240 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623886a8-1267-4c67-9732-c75c9a2b2a94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:59.130057 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623886a8-1267-4c67-9732-c75c9a2b2a94 None admin] RBAC: Adding query filter params (role.id=33c5141e078844bbb60127f2b26540d7, scope.project.id=459ff8f521284cf8b3ec858bfb1ba630, user.id=6636e4878e5846e598b80dcba8afbeef) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:34:59.130366 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623886a8-1267-4c67-9732-c75c9a2b2a94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:59.131852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623886a8-1267-4c67-9732-c75c9a2b2a94 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:59.140679 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 86/160] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:34:58 2022] GET /identity/v3/role_assignments?user.id=6636e4878e5846e598b80dcba8afbeef&scope.project.id=459ff8f521284cf8b3ec858bfb1ba630&role.id=33c5141e078844bbb60127f2b26540d7 => generated 610 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:34:59.146084 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d8897a-6691-45ae-821f-09ad1dc454ba None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:59.219911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ec3b0-72da-4702-a89e-4bb3aebdc274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:59.221749 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ec3b0-72da-4702-a89e-4bb3aebdc274 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:59.222409 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ec3b0-72da-4702-a89e-4bb3aebdc274 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:59.222916 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ec3b0-72da-4702-a89e-4bb3aebdc274 None admin] PATH_INFO: `/v3/users/4fb0c392df554f5dbd68f6861be954a1` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:59.224807 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ec3b0-72da-4702-a89e-4bb3aebdc274 None admin] RBAC: Authorizing `identity:get_user(user_id=4fb0c392df554f5dbd68f6861be954a1)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:59.226760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ec3b0-72da-4702-a89e-4bb3aebdc274 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:59.228790 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 75/161] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:34:58 2022] GET /identity/v3/users/4fb0c392df554f5dbd68f6861be954a1 => generated 248 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:59.234590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623886a8-1267-4c67-9732-c75c9a2b2a94 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:59.370264 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623886a8-1267-4c67-9732-c75c9a2b2a94 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:59.410741 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc88cb-f823-4d40-b75e-99af77841a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:59.411403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc88cb-f823-4d40-b75e-99af77841a8c None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:59.411681 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc88cb-f823-4d40-b75e-99af77841a8c None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:59.411952 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dc88cb-f823-4d40-b75e-99af77841a8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:59.447449 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:59.505024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dc88cb-f823-4d40-b75e-99af77841a8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:59.506534 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dc88cb-f823-4d40-b75e-99af77841a8c None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:59.640122 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 87/162] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:34:59 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:34:59.648195 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ec3b0-72da-4702-a89e-4bb3aebdc274 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:59.694740 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:34:59.784386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ec3b0-72da-4702-a89e-4bb3aebdc274 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:34:59.823826 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840d199-5de4-4cd3-aa89-86f36296ff0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:34:59.824859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840d199-5de4-4cd3-aa89-86f36296ff0c None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:59.825441 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840d199-5de4-4cd3-aa89-86f36296ff0c None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:59.825966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840d199-5de4-4cd3-aa89-86f36296ff0c None admin] PATH_INFO: `/v3/projects/459ff8f521284cf8b3ec858bfb1ba630` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:59.827484 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4840d199-5de4-4cd3-aa89-86f36296ff0c None admin] RBAC: Authorizing `identity:get_project(project_id=459ff8f521284cf8b3ec858bfb1ba630)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:34:59.829173 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4840d199-5de4-4cd3-aa89-86f36296ff0c None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:34:59.831696 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 76/163] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:34:59 2022] GET /identity/v3/projects/459ff8f521284cf8b3ec858bfb1ba630 => generated 299 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:34:59.838214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc88cb-f823-4d40-b75e-99af77841a8c None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:34:59.857299 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0878a-5e93-43e2-9b3b-546a28862549 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:59.857909 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0878a-5e93-43e2-9b3b-546a28862549 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:59.858427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f0878a-5e93-43e2-9b3b-546a28862549 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:59.860039 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 77/164] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:34:59 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:34:59.866378 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe76720-d3de-4ab2-b307-3f6f3f4015d9 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:34:59.866956 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe76720-d3de-4ab2-b307-3f6f3f4015d9 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:34:59.867462 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe76720-d3de-4ab2-b307-3f6f3f4015d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:34:59.898107 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-abe76720-d3de-4ab2-b307-3f6f3f4015d9 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:34:59.899258 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-abe76720-d3de-4ab2-b307-3f6f3f4015d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:34:59.944165 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:00.009455 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dc88cb-f823-4d40-b75e-99af77841a8c None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:00.052359 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 78/165] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:34:59 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:35:00.054322 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834bfff-1bef-44f7-b75b-dafc3b80d7c4 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:00.055617 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834bfff-1bef-44f7-b75b-dafc3b80d7c4 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:00.056267 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f834bfff-1bef-44f7-b75b-dafc3b80d7c4 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:00.059868 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 79/166] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:00 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:00.068492 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bde2c-83c7-44c1-93d4-e24f15fd03cd None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:00.069065 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bde2c-83c7-44c1-93d4-e24f15fd03cd None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:00.069559 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8bde2c-83c7-44c1-93d4-e24f15fd03cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:00.079455 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b487ad-1243-4aad-9276-cdd37d9dd7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:00.080476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b487ad-1243-4aad-9276-cdd37d9dd7c5 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:00.082481 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b487ad-1243-4aad-9276-cdd37d9dd7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:00.083324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b487ad-1243-4aad-9276-cdd37d9dd7c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:00.084422 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b487ad-1243-4aad-9276-cdd37d9dd7c5 None admin] RBAC: Adding query filter params (role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac, scope.project.id=459ff8f521284cf8b3ec858bfb1ba630, user.id=4fb0c392df554f5dbd68f6861be954a1) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:00.085780 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b487ad-1243-4aad-9276-cdd37d9dd7c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:00.094205 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b487ad-1243-4aad-9276-cdd37d9dd7c5 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:00.112436 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 88/167] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:34:59 2022] GET /identity/v3/role_assignments?user.id=4fb0c392df554f5dbd68f6861be954a1&scope.project.id=459ff8f521284cf8b3ec858bfb1ba630&role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 610 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:00.114511 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b487ad-1243-4aad-9276-cdd37d9dd7c5 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:00.115715 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-ee8bde2c-83c7-44c1-93d4-e24f15fd03cd None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:00.117775 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-ee8bde2c-83c7-44c1-93d4-e24f15fd03cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:00.167525 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:00.276127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 80/168] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:00 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:00.327040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b487ad-1243-4aad-9276-cdd37d9dd7c5 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:00.329073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8bde2c-83c7-44c1-93d4-e24f15fd03cd None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:00.395069 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:00.425929 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44a716-152a-4e25-8bf5-141c8a47a374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:00.426539 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44a716-152a-4e25-8bf5-141c8a47a374 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:00.426828 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44a716-152a-4e25-8bf5-141c8a47a374 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:00.427127 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb44a716-152a-4e25-8bf5-141c8a47a374 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:00.470805 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb44a716-152a-4e25-8bf5-141c8a47a374 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:00.483313 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-cb44a716-152a-4e25-8bf5-141c8a47a374 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:35:00.483313 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:00.483313 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:00.483313 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:00.483313 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:00.483313 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:00.483313 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:00.483313 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:00.483313 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:00.483313 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:00.483313 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:00.483313 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:00.483313 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:00.483313 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:00.483313 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:00.483313 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:00.484363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:00.484363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:00.484363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:00.484363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:00.484363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:00.484363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:00.484363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:00.484363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:00.484363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:00.484363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:00.484363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:00.484363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:00.484363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:00.484363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:00.484363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:00.484363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:00.485143 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:00.485143 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:00.485143 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:00.485143 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:00.485143 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:00.485143 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:00.485143 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:00.485143 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:00.485143 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:00.485143 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:00.485143 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:35:00.485143 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:00.488309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 89/169] 10.222.0.115 () {56 vars in 1119 bytes} [Tue Aug 2 10:35:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:00.495623 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44a716-152a-4e25-8bf5-141c8a47a374 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:00.573081 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8bde2c-83c7-44c1-93d4-e24f15fd03cd None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:00.640012 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b311a48d-6d5e-4c7d-8d5d-7edd25d125ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:00.640675 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311a48d-6d5e-4c7d-8d5d-7edd25d125ed None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:00.640960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311a48d-6d5e-4c7d-8d5d-7edd25d125ed None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:00.641234 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b311a48d-6d5e-4c7d-8d5d-7edd25d125ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:00.662851 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b311a48d-6d5e-4c7d-8d5d-7edd25d125ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:00.668761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-b311a48d-6d5e-4c7d-8d5d-7edd25d125ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:35:00.668761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:00.668761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:00.668761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:00.668761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:00.668761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:00.668761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:00.668761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:00.668761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:00.668761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:00.668761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:00.668761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:00.668761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:00.668761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:00.668761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:00.668761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:00.669657 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:00.669657 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:00.669657 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:00.669657 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:00.669657 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:00.669657 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:00.669657 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:00.669657 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:00.669657 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:00.669657 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:00.669657 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:00.669657 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:00.669657 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:00.669657 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:00.669657 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:00.669657 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:00.670437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:00.670437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:00.670437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:00.670437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:00.670437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:00.670437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:00.670437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:00.670437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:00.670437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:00.670437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:00.670437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:35:00.670437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:00.678048 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 81/170] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:35:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:35:00.681115 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb44a716-152a-4e25-8bf5-141c8a47a374 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:00.684423 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b311a48d-6d5e-4c7d-8d5d-7edd25d125ed None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:00.719545 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:00.734643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd9e45-81c1-48e5-af85-384f3ac9a0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:00.735294 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd9e45-81c1-48e5-af85-384f3ac9a0b1 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:00.735615 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd9e45-81c1-48e5-af85-384f3ac9a0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:00.735830 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bd9e45-81c1-48e5-af85-384f3ac9a0b1 None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:00.737180 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd9e45-81c1-48e5-af85-384f3ac9a0b1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:00.737480 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd9e45-81c1-48e5-af85-384f3ac9a0b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:00.738874 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bd9e45-81c1-48e5-af85-384f3ac9a0b1 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:00.757556 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 90/171] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:35:00 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:00.764500 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd9e45-81c1-48e5-af85-384f3ac9a0b1 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:00.840429 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b311a48d-6d5e-4c7d-8d5d-7edd25d125ed None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:00.849195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:00.897398 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98182f2-8524-45d4-8311-673575dcd26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:00.898522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98182f2-8524-45d4-8311-673575dcd26e None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:00.899203 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98182f2-8524-45d4-8311-673575dcd26e None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:00.900736 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98182f2-8524-45d4-8311-673575dcd26e None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:00.901916 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98182f2-8524-45d4-8311-673575dcd26e None admin] RBAC: Adding query filter params (name=member) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:00.902884 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98182f2-8524-45d4-8311-673575dcd26e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:00.905425 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98182f2-8524-45d4-8311-673575dcd26e None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:00.925244 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 82/172] 10.222.0.115 () {56 vars in 1106 bytes} [Tue Aug 2 10:35:00 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:35:00.930860 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98182f2-8524-45d4-8311-673575dcd26e None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:00.933795 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bd9e45-81c1-48e5-af85-384f3ac9a0b1 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:01.011174 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c1170b-b3dd-4cf1-8679-60f4f0cc4597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:01.011174 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1170b-b3dd-4cf1-8679-60f4f0cc4597 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:01.011174 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1170b-b3dd-4cf1-8679-60f4f0cc4597 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:01.011174 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c1170b-b3dd-4cf1-8679-60f4f0cc4597 None admin] PATH_INFO: `/v3/users/6636e4878e5846e598b80dcba8afbeef` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:01.015700 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c1170b-b3dd-4cf1-8679-60f4f0cc4597 None admin] RBAC: Authorizing `identity:get_user(user_id=6636e4878e5846e598b80dcba8afbeef)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:01.017228 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c1170b-b3dd-4cf1-8679-60f4f0cc4597 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:01.019671 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 91/173] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:35:00 2022] GET /identity/v3/users/6636e4878e5846e598b80dcba8afbeef => generated 276 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:01.020908 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11231d5-9893-4829-86e8-0a47e4605884 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:01.021192 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11231d5-9893-4829-86e8-0a47e4605884 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:01.021467 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11231d5-9893-4829-86e8-0a47e4605884 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:01.022715 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 92/174] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:01 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:01.034516 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c67b8-9fa2-4ddf-a70e-38b7a0f91ab2 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:01.034815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c67b8-9fa2-4ddf-a70e-38b7a0f91ab2 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:01.035089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c67b8-9fa2-4ddf-a70e-38b7a0f91ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:01.078631 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-757c67b8-9fa2-4ddf-a70e-38b7a0f91ab2 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:01.080062 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-757c67b8-9fa2-4ddf-a70e-38b7a0f91ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:01.086897 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:01.134976 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98182f2-8524-45d4-8311-673575dcd26e None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:01.209109 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d154fe-d98b-48e4-b9f0-28cff6303bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:01.211687 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d154fe-d98b-48e4-b9f0-28cff6303bf6 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:01.212450 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d154fe-d98b-48e4-b9f0-28cff6303bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:01.214135 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d154fe-d98b-48e4-b9f0-28cff6303bf6 None admin] PATH_INFO: `/v3/users/6636e4878e5846e598b80dcba8afbeef` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:01.217947 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 93/175] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:01 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:01.218382 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d154fe-d98b-48e4-b9f0-28cff6303bf6 None admin] RBAC: Authorizing `identity:get_user(user_id=6636e4878e5846e598b80dcba8afbeef)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:01.220073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d154fe-d98b-48e4-b9f0-28cff6303bf6 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:01.223127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 83/176] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:35:00 2022] GET /identity/v3/users/6636e4878e5846e598b80dcba8afbeef => generated 276 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:01.223586 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c67b8-9fa2-4ddf-a70e-38b7a0f91ab2 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:01.228783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d154fe-d98b-48e4-b9f0-28cff6303bf6 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:01.327416 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:01.426823 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d154fe-d98b-48e4-b9f0-28cff6303bf6 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:01.470599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c67b8-9fa2-4ddf-a70e-38b7a0f91ab2 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:01.508962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147c9ee-8e00-41f9-8483-2151982a9bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:01.509946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147c9ee-8e00-41f9-8483-2151982a9bdc None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:01.510421 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147c9ee-8e00-41f9-8483-2151982a9bdc None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:01.510892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0147c9ee-8e00-41f9-8483-2151982a9bdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:01.531282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:01.534136 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35ec058-0eb4-4f95-ae94-cef9a9ce0829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:01.535602 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ec058-0eb4-4f95-ae94-cef9a9ce0829 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:01.536099 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ec058-0eb4-4f95-ae94-cef9a9ce0829 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:01.536583 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35ec058-0eb4-4f95-ae94-cef9a9ce0829 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:01.604558 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0147c9ee-8e00-41f9-8483-2151982a9bdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:01.606051 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0147c9ee-8e00-41f9-8483-2151982a9bdc None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:01.623745 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35ec058-0eb4-4f95-ae94-cef9a9ce0829 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:01.625211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35ec058-0eb4-4f95-ae94-cef9a9ce0829 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:01.685749 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:01.733425 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 84/177] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:35:01 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:35:01.734757 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147c9ee-8e00-41f9-8483-2151982a9bdc None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:01.759641 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 94/178] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:35:01 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:35:01.761165 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76e96d-288b-4b44-adcd-e505d9f60dda None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:01.761448 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76e96d-288b-4b44-adcd-e505d9f60dda None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:01.761719 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e76e96d-288b-4b44-adcd-e505d9f60dda None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:01.762975 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 95/179] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:01.768348 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf31e3-74b4-47df-b094-686c3ffcc900 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:01.768348 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf31e3-74b4-47df-b094-686c3ffcc900 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:01.768348 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaf31e3-74b4-47df-b094-686c3ffcc900 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:01.768348 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 96/180] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:01.768348 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf31e3-74b4-47df-b094-686c3ffcc900 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:01.800411 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:01.864999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0147c9ee-8e00-41f9-8483-2151982a9bdc None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:01.905719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e015b0c-214e-4d23-a210-0b79d07116c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:01.906357 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e015b0c-214e-4d23-a210-0b79d07116c3 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:01.906638 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e015b0c-214e-4d23-a210-0b79d07116c3 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:01.906905 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e015b0c-214e-4d23-a210-0b79d07116c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:01.918559 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e015b0c-214e-4d23-a210-0b79d07116c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:01.923680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-5e015b0c-214e-4d23-a210-0b79d07116c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:35:01.923680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:01.923680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:01.923680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:01.923680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:01.923680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:01.923680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:01.923680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:01.923680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:01.923680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:01.923680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:01.923680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:01.923680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:01.923680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:01.923680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:01.923680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:01.929922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:01.929922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:01.929922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:01.929922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:01.929922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:01.929922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:01.929922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:01.929922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:01.929922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:01.929922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:01.929922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:01.929922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:01.929922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:01.929922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:01.929922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:01.929922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:01.930742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:01.930742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:01.930742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:01.930742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:01.930742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:01.930742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:01.930742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:01.930742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:01.930742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:01.930742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:01.930742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:35:01.930742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:01.930742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 85/181] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:35:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:01.930742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e015b0c-214e-4d23-a210-0b79d07116c3 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:01.931498 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaf31e3-74b4-47df-b094-686c3ffcc900 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:01.967495 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a69e23b-c618-4bd0-8bd1-94f05c6b799e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:01.968163 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69e23b-c618-4bd0-8bd1-94f05c6b799e None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:01.968439 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69e23b-c618-4bd0-8bd1-94f05c6b799e None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:01.968712 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a69e23b-c618-4bd0-8bd1-94f05c6b799e None admin] PATH_INFO: `/v3/projects/009a328c01ce4fed9c5b90fec435b08f` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:01.969882 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a69e23b-c618-4bd0-8bd1-94f05c6b799e None admin] RBAC: Authorizing `identity:get_project(project_id=009a328c01ce4fed9c5b90fec435b08f)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:01.971412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a69e23b-c618-4bd0-8bd1-94f05c6b799e None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:01.973263 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 97/182] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:35:01 2022] GET /identity/v3/projects/009a328c01ce4fed9c5b90fec435b08f => generated 313 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:01.974345 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9eb2bc-8d9d-4afa-806d-8793e8d4111a None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:01.974636 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9eb2bc-8d9d-4afa-806d-8793e8d4111a None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:01.974910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9eb2bc-8d9d-4afa-806d-8793e8d4111a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:01.993152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-5a9eb2bc-8d9d-4afa-806d-8793e8d4111a None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:01.994258 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-5a9eb2bc-8d9d-4afa-806d-8793e8d4111a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:02.002418 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:02.033588 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e015b0c-214e-4d23-a210-0b79d07116c3 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:02.041032 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:02.061682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 98/183] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:01 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:02.064706 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8bbb6-8ae0-473a-8ca2-57cd59cd4633 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:02.065001 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8bbb6-8ae0-473a-8ca2-57cd59cd4633 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:02.065280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d8bbb6-8ae0-473a-8ca2-57cd59cd4633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:02.073749 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f0b582-bf3b-4ff2-b6ba-81fca079638e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:02.074406 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0b582-bf3b-4ff2-b6ba-81fca079638e None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:02.075009 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0b582-bf3b-4ff2-b6ba-81fca079638e None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:02.075318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f0b582-bf3b-4ff2-b6ba-81fca079638e None admin] PATH_INFO: `/v3/projects/459ff8f521284cf8b3ec858bfb1ba630` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:02.076543 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f0b582-bf3b-4ff2-b6ba-81fca079638e None admin] RBAC: Authorizing `identity:get_project(project_id=459ff8f521284cf8b3ec858bfb1ba630)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:02.078499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f0b582-bf3b-4ff2-b6ba-81fca079638e None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:02.081038 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 86/184] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:35:01 2022] GET /identity/v3/projects/459ff8f521284cf8b3ec858bfb1ba630 => generated 299 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:02.082439 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04dfb65-a23b-4833-ae90-d5bb96a36fc5 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:02.082722 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04dfb65-a23b-4833-ae90-d5bb96a36fc5 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:02.083041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04dfb65-a23b-4833-ae90-d5bb96a36fc5 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:02.084494 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 87/185] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:02.089160 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04dfb65-a23b-4833-ae90-d5bb96a36fc5 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:02.092436 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-f5d8bbb6-8ae0-473a-8ca2-57cd59cd4633 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:02.093635 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-f5d8bbb6-8ae0-473a-8ca2-57cd59cd4633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:02.160700 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 99/186] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:02 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:02.161627 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d8bbb6-8ae0-473a-8ca2-57cd59cd4633 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:02.194484 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04dfb65-a23b-4833-ae90-d5bb96a36fc5 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:02.230245 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94d94b-dfb8-42de-87bd-057f213c0c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:02.231300 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94d94b-dfb8-42de-87bd-057f213c0c56 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:02.231661 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94d94b-dfb8-42de-87bd-057f213c0c56 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:02.231967 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db94d94b-dfb8-42de-87bd-057f213c0c56 None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:02.232681 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db94d94b-dfb8-42de-87bd-057f213c0c56 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:02.232989 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db94d94b-dfb8-42de-87bd-057f213c0c56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:02.234418 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db94d94b-dfb8-42de-87bd-057f213c0c56 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:02.243117 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 88/187] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:35:02 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:02.243993 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94d94b-dfb8-42de-87bd-057f213c0c56 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:02.253655 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:02.279256 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:02.284127 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d8bbb6-8ae0-473a-8ca2-57cd59cd4633 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:02.318961 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e490313d-9e20-4380-8c76-9aa37ff0e03a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:02.319603 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490313d-9e20-4380-8c76-9aa37ff0e03a None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:02.319883 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490313d-9e20-4380-8c76-9aa37ff0e03a None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:02.320156 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e490313d-9e20-4380-8c76-9aa37ff0e03a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:02.320949 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e490313d-9e20-4380-8c76-9aa37ff0e03a None admin] RBAC: Adding query filter params (role.id=33c5141e078844bbb60127f2b26540d7, scope.project.id=009a328c01ce4fed9c5b90fec435b08f, user.id=6636e4878e5846e598b80dcba8afbeef) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:02.321257 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e490313d-9e20-4380-8c76-9aa37ff0e03a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:02.322747 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e490313d-9e20-4380-8c76-9aa37ff0e03a None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:02.328666 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 100/188] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:35:02 2022] GET /identity/v3/role_assignments?user.id=6636e4878e5846e598b80dcba8afbeef&scope.project.id=009a328c01ce4fed9c5b90fec435b08f&role.id=33c5141e078844bbb60127f2b26540d7 => generated 610 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:02.329445 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e490313d-9e20-4380-8c76-9aa37ff0e03a None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:02.351065 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db94d94b-dfb8-42de-87bd-057f213c0c56 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:02.393681 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c2a8e9-d6fa-42ae-ad44-0932c4a1495a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:02.394555 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2a8e9-d6fa-42ae-ad44-0932c4a1495a None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:02.394851 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2a8e9-d6fa-42ae-ad44-0932c4a1495a None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:02.395150 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2a8e9-d6fa-42ae-ad44-0932c4a1495a None admin] PATH_INFO: `/v3/domains/default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:02.396395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c2a8e9-d6fa-42ae-ad44-0932c4a1495a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:02.397884 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c2a8e9-d6fa-42ae-ad44-0932c4a1495a None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:02.399644 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 89/189] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:02 2022] GET /identity/v3/domains/default => generated 200 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:02.400912 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce61b6-173c-406f-9af0-e9ce6672cf18 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:02.401196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce61b6-173c-406f-9af0-e9ce6672cf18 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:02.401470 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce61b6-173c-406f-9af0-e9ce6672cf18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:02.421601 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-90ce61b6-173c-406f-9af0-e9ce6672cf18 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:02.422995 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-90ce61b6-173c-406f-9af0-e9ce6672cf18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:02.467318 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e490313d-9e20-4380-8c76-9aa37ff0e03a None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:02.503179 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 90/190] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:02 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:02.504309 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce61b6-173c-406f-9af0-e9ce6672cf18 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:02.515651 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5914d5cf-713f-423c-8da3-e72e4a888434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:02.516284 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914d5cf-713f-423c-8da3-e72e4a888434 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:02.516609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914d5cf-713f-423c-8da3-e72e4a888434 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:02.516990 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5914d5cf-713f-423c-8da3-e72e4a888434 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:02.518475 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5914d5cf-713f-423c-8da3-e72e4a888434 None admin] RBAC: Adding query filter params (role.id=a4213d713b6f49d78d7bbb9484c9312f, scope.project.id=459ff8f521284cf8b3ec858bfb1ba630, user.id=6636e4878e5846e598b80dcba8afbeef) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:02.518475 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5914d5cf-713f-423c-8da3-e72e4a888434 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:02.519887 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5914d5cf-713f-423c-8da3-e72e4a888434 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:02.523122 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:02.527713 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 101/191] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:35:02 2022] GET /identity/v3/role_assignments?user.id=6636e4878e5846e598b80dcba8afbeef&scope.project.id=459ff8f521284cf8b3ec858bfb1ba630&role.id=a4213d713b6f49d78d7bbb9484c9312f => generated 610 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:02.529315 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5914d5cf-713f-423c-8da3-e72e4a888434 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:02.547445 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:02.643464 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce61b6-173c-406f-9af0-e9ce6672cf18 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:02.679811 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5914d5cf-713f-423c-8da3-e72e4a888434 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:02.686210 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55ede7-fa3d-4e24-b157-0d122bcf8f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:02.686860 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55ede7-fa3d-4e24-b157-0d122bcf8f54 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:02.687751 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55ede7-fa3d-4e24-b157-0d122bcf8f54 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:02.688081 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a55ede7-fa3d-4e24-b157-0d122bcf8f54 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:02.701054 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a55ede7-fa3d-4e24-b157-0d122bcf8f54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:02.706628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-8a55ede7-fa3d-4e24-b157-0d122bcf8f54 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:35:02.706628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:02.706628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:02.706628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:02.706628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:02.706628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:02.706628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:02.706628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:02.706628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:02.706628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:02.706628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:02.706628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:02.706628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:02.706628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:02.706628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:02.706628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:02.707979 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:02.707979 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:02.707979 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:02.707979 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:02.707979 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:02.707979 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:02.707979 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:02.707979 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:02.707979 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:02.707979 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:02.707979 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:02.707979 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:02.707979 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:02.707979 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:02.707979 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:02.707979 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:02.708792 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:02.708792 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:02.708792 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:02.708792 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:02.708792 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:02.708792 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:02.708792 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:02.708792 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:02.708792 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:02.708792 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:02.708792 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:35:02.708792 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:02.709398 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 91/192] 10.222.0.115 () {56 vars in 1119 bytes} [Tue Aug 2 10:35:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:02.710323 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55ede7-fa3d-4e24-b157-0d122bcf8f54 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:02.725045 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6a7fd-be5d-4cfb-a1de-98ea5fd9b219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:02.725709 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6a7fd-be5d-4cfb-a1de-98ea5fd9b219 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:02.725992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6a7fd-be5d-4cfb-a1de-98ea5fd9b219 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:02.726279 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e6a7fd-be5d-4cfb-a1de-98ea5fd9b219 None admin] PATH_INFO: `/v3/users/544ca40877f949e6a646197c40282751` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:02.747065 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6a7fd-be5d-4cfb-a1de-98ea5fd9b219 None admin] RBAC: Authorizing `identity:get_user(user_id=544ca40877f949e6a646197c40282751)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:02.748564 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e6a7fd-be5d-4cfb-a1de-98ea5fd9b219 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:02.751111 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 102/193] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:35:02 2022] GET /identity/v3/users/544ca40877f949e6a646197c40282751 => generated 284 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:02.751111 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6a7fd-be5d-4cfb-a1de-98ea5fd9b219 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:02.813857 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:02.821194 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:02.827768 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a55ede7-fa3d-4e24-b157-0d122bcf8f54 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:02.857469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e6a7fd-be5d-4cfb-a1de-98ea5fd9b219 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:02.862337 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780034bb-cb3e-4924-9251-61de93cca7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:02.862950 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780034bb-cb3e-4924-9251-61de93cca7d8 None admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:02.863324 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780034bb-cb3e-4924-9251-61de93cca7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:02.863938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780034bb-cb3e-4924-9251-61de93cca7d8 None admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:02.864451 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780034bb-cb3e-4924-9251-61de93cca7d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:02.865918 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780034bb-cb3e-4924-9251-61de93cca7d8 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:02.894564 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b8ed39-658e-4453-8262-f631543d05fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:02.895267 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b8ed39-658e-4453-8262-f631543d05fc None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:02.895545 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b8ed39-658e-4453-8262-f631543d05fc None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:02.895814 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b8ed39-658e-4453-8262-f631543d05fc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:02.905910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b8ed39-658e-4453-8262-f631543d05fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:02.911980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-56b8ed39-658e-4453-8262-f631543d05fc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:35:02.911980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:02.911980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:02.911980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:02.911980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:02.911980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:02.911980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:02.911980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:02.911980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:02.911980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:02.911980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:02.911980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:02.911980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:02.911980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:02.911980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:02.911980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:02.913664 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:02.913664 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:02.913664 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:02.913664 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:02.913664 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:02.913664 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:02.913664 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:02.913664 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:02.913664 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:02.913664 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:02.913664 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:02.913664 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:02.913664 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:02.913664 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:02.913664 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:02.913664 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:02.914642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:02.914642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:02.914642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:02.914642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:02.914642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:02.914642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:02.914642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:02.914642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:02.914642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:02.914642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:02.914642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:35:02.914642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:02.914642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 103/194] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:35:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:02.915377 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b8ed39-658e-4453-8262-f631543d05fc None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:03.000968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 92/195] 10.222.0.115 () {60 vars in 1137 bytes} [Tue Aug 2 10:35:02 2022] POST /identity/v3/users => generated 298 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:35:03.007288 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780034bb-cb3e-4924-9251-61de93cca7d8 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:03.020266 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b8ed39-658e-4453-8262-f631543d05fc None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:03.042626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:03.063040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe33bf5-67ab-41c2-9348-ccb6c1516a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:03.063696 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe33bf5-67ab-41c2-9348-ccb6c1516a29 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:03.064062 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe33bf5-67ab-41c2-9348-ccb6c1516a29 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:03.064285 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe33bf5-67ab-41c2-9348-ccb6c1516a29 None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:03.065010 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe33bf5-67ab-41c2-9348-ccb6c1516a29 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:03.065427 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe33bf5-67ab-41c2-9348-ccb6c1516a29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:03.067139 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe33bf5-67ab-41c2-9348-ccb6c1516a29 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:03.076500 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 104/196] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:35:02 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:03.077304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe33bf5-67ab-41c2-9348-ccb6c1516a29 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:03.127593 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780034bb-cb3e-4924-9251-61de93cca7d8 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:03.176695 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7c1bd-7b6d-4410-b071-189f017e786d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:03.177336 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7c1bd-7b6d-4410-b071-189f017e786d None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:03.177617 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7c1bd-7b6d-4410-b071-189f017e786d None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:03.177895 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7c1bd-7b6d-4410-b071-189f017e786d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:03.194757 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe33bf5-67ab-41c2-9348-ccb6c1516a29 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:03.200162 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:03.244719 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851ab33-7977-4788-b54b-299ada07afe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:03.245357 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851ab33-7977-4788-b54b-299ada07afe6 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:03.245639 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851ab33-7977-4788-b54b-299ada07afe6 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:03.245981 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b851ab33-7977-4788-b54b-299ada07afe6 None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:03.246710 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851ab33-7977-4788-b54b-299ada07afe6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:03.247017 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851ab33-7977-4788-b54b-299ada07afe6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:03.248979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b851ab33-7977-4788-b54b-299ada07afe6 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:03.260647 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 105/197] 10.222.0.115 () {56 vars in 1106 bytes} [Tue Aug 2 10:35:03 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:03.261225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851ab33-7977-4788-b54b-299ada07afe6 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:03.264453 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7c1bd-7b6d-4410-b071-189f017e786d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:03.266025 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7c1bd-7b6d-4410-b071-189f017e786d None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:03.319004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:03.394758 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b851ab33-7977-4788-b54b-299ada07afe6 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:03.396621 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 93/198] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:35:03 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:35:03.397603 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7c1bd-7b6d-4410-b071-189f017e786d None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:03.439635 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1176ae-a500-4971-a5c7-e665ac57221c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:03.440400 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1176ae-a500-4971-a5c7-e665ac57221c None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:03.440738 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1176ae-a500-4971-a5c7-e665ac57221c None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:03.441017 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1176ae-a500-4971-a5c7-e665ac57221c None admin] PATH_INFO: `/v3/users/544ca40877f949e6a646197c40282751` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:03.442272 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1176ae-a500-4971-a5c7-e665ac57221c None admin] RBAC: Authorizing `identity:get_user(user_id=544ca40877f949e6a646197c40282751)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:03.443959 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1176ae-a500-4971-a5c7-e665ac57221c None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:03.445920 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 106/199] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:35:03 2022] GET /identity/v3/users/544ca40877f949e6a646197c40282751 => generated 284 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:03.446823 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1176ae-a500-4971-a5c7-e665ac57221c None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:03.454485 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:03.526297 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7c1bd-7b6d-4410-b071-189f017e786d None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:03.564906 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca609e3-6dae-4613-869b-af3b943d8b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:03.565564 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca609e3-6dae-4613-869b-af3b943d8b7f None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:03.565843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca609e3-6dae-4613-869b-af3b943d8b7f None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:03.566116 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca609e3-6dae-4613-869b-af3b943d8b7f None admin] PATH_INFO: `/v3/users/35dea9b8dc7d4cc2b2e739876cc41d55` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:03.567482 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca609e3-6dae-4613-869b-af3b943d8b7f None admin] RBAC: Authorizing `identity:get_user(user_id=35dea9b8dc7d4cc2b2e739876cc41d55)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:03.569115 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca609e3-6dae-4613-869b-af3b943d8b7f None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:03.570891 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 94/200] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:35:03 2022] GET /identity/v3/users/35dea9b8dc7d4cc2b2e739876cc41d55 => generated 290 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:03.576406 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca609e3-6dae-4613-869b-af3b943d8b7f None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:03.590207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1176ae-a500-4971-a5c7-e665ac57221c None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:03.603187 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:03.630347 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e64a325-a519-4ba3-9c72-4b90798bca65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:03.631429 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64a325-a519-4ba3-9c72-4b90798bca65 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:03.631819 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64a325-a519-4ba3-9c72-4b90798bca65 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:03.632294 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e64a325-a519-4ba3-9c72-4b90798bca65 None admin] PATH_INFO: `/v3/projects/21b65aad6f794b3588a867ef02e946ac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:03.633848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e64a325-a519-4ba3-9c72-4b90798bca65 None admin] RBAC: Authorizing `identity:get_project(project_id=21b65aad6f794b3588a867ef02e946ac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:03.635623 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e64a325-a519-4ba3-9c72-4b90798bca65 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:03.638003 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 107/201] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:35:03 2022] GET /identity/v3/projects/21b65aad6f794b3588a867ef02e946ac => generated 303 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:03.639082 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e64a325-a519-4ba3-9c72-4b90798bca65 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:03.695397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca609e3-6dae-4613-869b-af3b943d8b7f None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:03.716088 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:03.734777 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c9683-e424-4437-b2ba-8fd1ce308ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:03.735727 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c9683-e424-4437-b2ba-8fd1ce308ff1 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:03.737381 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c9683-e424-4437-b2ba-8fd1ce308ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:03.737806 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1c9683-e424-4437-b2ba-8fd1ce308ff1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:03.754780 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e64a325-a519-4ba3-9c72-4b90798bca65 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:03.794788 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48fe105-1936-413c-946a-b5f4c48428c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:03.795621 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fe105-1936-413c-946a-b5f4c48428c2 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:03.796004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fe105-1936-413c-946a-b5f4c48428c2 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:03.796381 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48fe105-1936-413c-946a-b5f4c48428c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:03.814647 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1c9683-e424-4437-b2ba-8fd1ce308ff1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:03.816364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1c9683-e424-4437-b2ba-8fd1ce308ff1 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:03.848672 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:03.872897 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48fe105-1936-413c-946a-b5f4c48428c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:03.874702 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48fe105-1936-413c-946a-b5f4c48428c2 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:03.927733 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 95/202] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:35:03 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:35:03.929025 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c9683-e424-4437-b2ba-8fd1ce308ff1 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:03.949752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:03.995457 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 108/203] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:35:03 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:35:03.996590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48fe105-1936-413c-946a-b5f4c48428c2 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:04.050026 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1c9683-e424-4437-b2ba-8fd1ce308ff1 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:04.084819 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:04.089064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a11d1e-5d47-436d-8672-19cad14d7e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:04.089704 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a11d1e-5d47-436d-8672-19cad14d7e50 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:04.089985 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a11d1e-5d47-436d-8672-19cad14d7e50 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:04.090266 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a11d1e-5d47-436d-8672-19cad14d7e50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:04.091236 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a11d1e-5d47-436d-8672-19cad14d7e50 None admin] RBAC: Adding query filter params (role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac, scope.project.id=21b65aad6f794b3588a867ef02e946ac, user.id=544ca40877f949e6a646197c40282751) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:04.097783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a11d1e-5d47-436d-8672-19cad14d7e50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:04.097783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a11d1e-5d47-436d-8672-19cad14d7e50 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:04.107583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 96/204] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:35:03 2022] GET /identity/v3/role_assignments?user.id=544ca40877f949e6a646197c40282751&scope.project.id=21b65aad6f794b3588a867ef02e946ac&role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:04.108508 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a11d1e-5d47-436d-8672-19cad14d7e50 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:04.109449 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48fe105-1936-413c-946a-b5f4c48428c2 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:04.154009 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca909867-cae1-497a-809a-7248a2e7245f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:04.154694 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca909867-cae1-497a-809a-7248a2e7245f None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:04.154976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca909867-cae1-497a-809a-7248a2e7245f None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:04.155292 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca909867-cae1-497a-809a-7248a2e7245f None admin] PATH_INFO: `/v3/projects/21b65aad6f794b3588a867ef02e946ac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:04.156712 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca909867-cae1-497a-809a-7248a2e7245f None admin] RBAC: Authorizing `identity:get_project(project_id=21b65aad6f794b3588a867ef02e946ac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:04.158872 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca909867-cae1-497a-809a-7248a2e7245f None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:04.161109 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 109/205] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:35:03 2022] GET /identity/v3/projects/21b65aad6f794b3588a867ef02e946ac => generated 303 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:04.167248 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca909867-cae1-497a-809a-7248a2e7245f None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:04.286968 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca909867-cae1-497a-809a-7248a2e7245f None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:04.297976 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:04.329683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b93f39-1739-4c71-b1e5-ba36760e4fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:04.330873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b93f39-1739-4c71-b1e5-ba36760e4fb4 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:04.331189 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b93f39-1739-4c71-b1e5-ba36760e4fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:04.331464 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b93f39-1739-4c71-b1e5-ba36760e4fb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:04.332556 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b93f39-1739-4c71-b1e5-ba36760e4fb4 None admin] RBAC: Adding query filter params (role.id=a4213d713b6f49d78d7bbb9484c9312f, scope.project.id=21b65aad6f794b3588a867ef02e946ac, user.id=544ca40877f949e6a646197c40282751) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:04.332871 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b93f39-1739-4c71-b1e5-ba36760e4fb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:04.334386 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b93f39-1739-4c71-b1e5-ba36760e4fb4 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:04.342112 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 110/206] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:35:04 2022] GET /identity/v3/role_assignments?user.id=544ca40877f949e6a646197c40282751&scope.project.id=21b65aad6f794b3588a867ef02e946ac&role.id=a4213d713b6f49d78d7bbb9484c9312f => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:04.349186 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a11d1e-5d47-436d-8672-19cad14d7e50 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:04.418763 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e679b44-a1df-4a85-9872-79e3023e6ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:04.419403 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e679b44-a1df-4a85-9872-79e3023e6ced None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:04.419685 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e679b44-a1df-4a85-9872-79e3023e6ced None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:04.419966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e679b44-a1df-4a85-9872-79e3023e6ced None admin] PATH_INFO: `/v3/projects/459ff8f521284cf8b3ec858bfb1ba630` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:04.421311 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e679b44-a1df-4a85-9872-79e3023e6ced None admin] RBAC: Authorizing `identity:get_project(project_id=459ff8f521284cf8b3ec858bfb1ba630)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:04.422853 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e679b44-a1df-4a85-9872-79e3023e6ced None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:04.425310 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 97/207] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:35:04 2022] GET /identity/v3/projects/459ff8f521284cf8b3ec858bfb1ba630 => generated 299 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:04.432260 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b93f39-1739-4c71-b1e5-ba36760e4fb4 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:04.466038 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:04.594603 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b93f39-1739-4c71-b1e5-ba36760e4fb4 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:04.633792 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e9738f-fb78-4c8a-8154-620a3ad635f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:04.634437 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9738f-fb78-4c8a-8154-620a3ad635f4 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:04.634724 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9738f-fb78-4c8a-8154-620a3ad635f4 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:04.635002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e9738f-fb78-4c8a-8154-620a3ad635f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:04.635867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9738f-fb78-4c8a-8154-620a3ad635f4 None admin] RBAC: Adding query filter params (role.id=1cf433b432764fbab711f54dcf7aa81e, scope.project.id=459ff8f521284cf8b3ec858bfb1ba630, user.id=35dea9b8dc7d4cc2b2e739876cc41d55) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:04.636182 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9738f-fb78-4c8a-8154-620a3ad635f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:04.637640 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e9738f-fb78-4c8a-8154-620a3ad635f4 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:04.646026 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 111/208] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:35:04 2022] GET /identity/v3/role_assignments?user.id=35dea9b8dc7d4cc2b2e739876cc41d55&scope.project.id=459ff8f521284cf8b3ec858bfb1ba630&role.id=1cf433b432764fbab711f54dcf7aa81e => generated 610 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:35:05.433562 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90956e84-ea8c-4136-b8f2-c8b2ecee3cd5 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:05.434073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90956e84-ea8c-4136-b8f2-c8b2ecee3cd5 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:05.434179 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90956e84-ea8c-4136-b8f2-c8b2ecee3cd5 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:05.435528 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 98/209] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:05.441646 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9048f74-e2f9-457f-8027-f0f96e56a80d None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:05.442107 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9048f74-e2f9-457f-8027-f0f96e56a80d None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:05.442184 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9048f74-e2f9-457f-8027-f0f96e56a80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:05.462065 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-e9048f74-e2f9-457f-8027-f0f96e56a80d None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:05.463543 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-e9048f74-e2f9-457f-8027-f0f96e56a80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:05.475470 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003559a-b478-4c57-aa97-f2dfbbd178ba None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:05.476229 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003559a-b478-4c57-aa97-f2dfbbd178ba None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:05.476730 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003559a-b478-4c57-aa97-f2dfbbd178ba None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:05.478333 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 99/210] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:05 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:05.490589 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0cba30-e9cc-4c28-ac0b-d53e95137218 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:05.491247 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0cba30-e9cc-4c28-ac0b-d53e95137218 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:05.491523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0cba30-e9cc-4c28-ac0b-d53e95137218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:05.514261 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-3d0cba30-e9cc-4c28-ac0b-d53e95137218 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:05.516261 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-3d0cba30-e9cc-4c28-ac0b-d53e95137218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:05.553844 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 112/211] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:05 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:05.581887 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9048f74-e2f9-457f-8027-f0f96e56a80d None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:05.599648 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:05.603410 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 100/212] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:05 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:35:05.627913 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0cba30-e9cc-4c28-ac0b-d53e95137218 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:05.722038 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9048f74-e2f9-457f-8027-f0f96e56a80d None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:05.723365 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:05.746148 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0cba30-e9cc-4c28-ac0b-d53e95137218 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:05.763179 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7696d50f-5024-4628-b1d4-e1a1a039c22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:05.763825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696d50f-5024-4628-b1d4-e1a1a039c22b None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:05.764101 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696d50f-5024-4628-b1d4-e1a1a039c22b None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:05.764373 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7696d50f-5024-4628-b1d4-e1a1a039c22b None admin] PATH_INFO: `/v3/roles/member` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:05.777597 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7696d50f-5024-4628-b1d4-e1a1a039c22b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:05.782877 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-7696d50f-5024-4628-b1d4-e1a1a039c22b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:35:05.782877 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:05.782877 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:05.782877 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:05.782877 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:05.782877 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:05.782877 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:05.782877 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:05.782877 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:05.782877 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:05.782877 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:05.782877 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:05.782877 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:05.782877 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:05.782877 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:05.782877 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:05.783964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:05.783964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:05.783964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:05.783964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:05.783964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:05.783964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:05.783964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:05.783964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:05.783964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:05.783964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:05.783964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:05.783964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:05.783964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:05.783964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:05.783964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:05.783964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:05.784840 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:05.784840 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:05.784840 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:05.784840 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:05.784840 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:05.784840 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:05.784840 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:05.784840 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:05.784840 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:05.784840 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:05.784840 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:35:05.784840 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:05.789649 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 113/213] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:35:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:35:05.790607 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7696d50f-5024-4628-b1d4-e1a1a039c22b None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:05.808146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735b1d0-cada-4d28-9a59-5de2cfd3ae6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:05.809183 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735b1d0-cada-4d28-9a59-5de2cfd3ae6f None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:05.809663 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735b1d0-cada-4d28-9a59-5de2cfd3ae6f None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:05.810148 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735b1d0-cada-4d28-9a59-5de2cfd3ae6f None admin] PATH_INFO: `/v3/domains/default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:05.811578 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f735b1d0-cada-4d28-9a59-5de2cfd3ae6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:05.813241 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f735b1d0-cada-4d28-9a59-5de2cfd3ae6f None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:05.816522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 101/214] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:05 2022] GET /identity/v3/domains/default => generated 200 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:35:05.821661 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735b1d0-cada-4d28-9a59-5de2cfd3ae6f None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:05.889034 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:05.930944 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7696d50f-5024-4628-b1d4-e1a1a039c22b None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:05.937488 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735b1d0-cada-4d28-9a59-5de2cfd3ae6f None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:05.970370 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec38e6b-a2e5-4700-8ef2-e7a1443e7aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:05.971420 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec38e6b-a2e5-4700-8ef2-e7a1443e7aab None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:05.972007 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec38e6b-a2e5-4700-8ef2-e7a1443e7aab None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:05.972635 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec38e6b-a2e5-4700-8ef2-e7a1443e7aab None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:05.973618 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec38e6b-a2e5-4700-8ef2-e7a1443e7aab None admin] RBAC: Adding query filter params (name=member) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:05.974138 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec38e6b-a2e5-4700-8ef2-e7a1443e7aab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:05.975830 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec38e6b-a2e5-4700-8ef2-e7a1443e7aab None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:05.986750 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e14347-b57a-4058-98f9-ccb3f0a47807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:05.988297 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e14347-b57a-4058-98f9-ccb3f0a47807 None admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:05.988576 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e14347-b57a-4058-98f9-ccb3f0a47807 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:05.988849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e14347-b57a-4058-98f9-ccb3f0a47807 None admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:05.989852 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e14347-b57a-4058-98f9-ccb3f0a47807 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:05.991363 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e14347-b57a-4058-98f9-ccb3f0a47807 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:05.992816 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 114/215] 10.222.0.115 () {56 vars in 1106 bytes} [Tue Aug 2 10:35:05 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:05.998247 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec38e6b-a2e5-4700-8ef2-e7a1443e7aab None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:06.093523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 102/216] 10.222.0.115 () {60 vars in 1137 bytes} [Tue Aug 2 10:35:05 2022] POST /identity/v3/users => generated 298 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:35:06.118283 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec38e6b-a2e5-4700-8ef2-e7a1443e7aab None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:06.148752 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:06.157449 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f394c-d248-4f78-8586-ffaa5194fb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:06.158654 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f394c-d248-4f78-8586-ffaa5194fb19 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:06.158941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f394c-d248-4f78-8586-ffaa5194fb19 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:06.159265 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f394c-d248-4f78-8586-ffaa5194fb19 None admin] PATH_INFO: `/v3/users/06535fb063c04013957d150e411bec79` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:06.176177 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93f394c-d248-4f78-8586-ffaa5194fb19 None admin] RBAC: Authorizing `identity:get_user(user_id=06535fb063c04013957d150e411bec79)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:06.177701 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93f394c-d248-4f78-8586-ffaa5194fb19 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:06.179492 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 115/217] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:35:05 2022] GET /identity/v3/users/06535fb063c04013957d150e411bec79 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:35:06.185108 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e14347-b57a-4058-98f9-ccb3f0a47807 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:06.189194 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:06.298173 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e14347-b57a-4058-98f9-ccb3f0a47807 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:06.337913 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d33ad0c-f26f-44ea-afda-523613d0513d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:06.338534 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33ad0c-f26f-44ea-afda-523613d0513d None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:06.338815 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33ad0c-f26f-44ea-afda-523613d0513d None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:06.339119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d33ad0c-f26f-44ea-afda-523613d0513d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:06.415164 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d33ad0c-f26f-44ea-afda-523613d0513d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:06.416764 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d33ad0c-f26f-44ea-afda-523613d0513d None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:06.420043 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:06.537658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 103/218] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:35:06 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:35:06.543450 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f394c-d248-4f78-8586-ffaa5194fb19 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:06.565782 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2050428-cf36-471a-8a4c-d62e1349e481 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:06.566015 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2050428-cf36-471a-8a4c-d62e1349e481 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:06.566230 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2050428-cf36-471a-8a4c-d62e1349e481 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:06.567637 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 104/219] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:06 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:35:06.573074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5f5f0-7bee-4dbd-8403-980886c407b7 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:06.573358 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5f5f0-7bee-4dbd-8403-980886c407b7 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:06.573633 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b5f5f0-7bee-4dbd-8403-980886c407b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:06.594073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-a3b5f5f0-7bee-4dbd-8403-980886c407b7 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:06.595834 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-a3b5f5f0-7bee-4dbd-8403-980886c407b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:06.676172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 105/220] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:06 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:35:06.701841 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b5f5f0-7bee-4dbd-8403-980886c407b7 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:06.703353 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f394c-d248-4f78-8586-ffaa5194fb19 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:06.743484 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:06.774148 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb5bc4-8f73-478d-aa28-925f37ae2cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:06.774805 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb5bc4-8f73-478d-aa28-925f37ae2cc6 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:06.775121 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb5bc4-8f73-478d-aa28-925f37ae2cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:06.775406 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb5bc4-8f73-478d-aa28-925f37ae2cc6 None admin] PATH_INFO: `/v3/projects/21b65aad6f794b3588a867ef02e946ac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:06.777185 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cb5bc4-8f73-478d-aa28-925f37ae2cc6 None admin] RBAC: Authorizing `identity:get_project(project_id=21b65aad6f794b3588a867ef02e946ac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:06.778694 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cb5bc4-8f73-478d-aa28-925f37ae2cc6 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:06.780694 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 116/221] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:35:06 2022] GET /identity/v3/projects/21b65aad6f794b3588a867ef02e946ac => generated 303 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:06.781724 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9087e-3740-49f5-aa85-0631d5a7df91 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:06.782012 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9087e-3740-49f5-aa85-0631d5a7df91 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:06.782291 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9087e-3740-49f5-aa85-0631d5a7df91 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:06.783915 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 117/222] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:06 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:06.789070 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9087e-3740-49f5-aa85-0631d5a7df91 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:06.850235 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b5f5f0-7bee-4dbd-8403-980886c407b7 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:06.858693 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:06.889631 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391dd84f-802c-472a-a602-b456e00920d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:06.890471 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391dd84f-802c-472a-a602-b456e00920d0 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:06.890755 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391dd84f-802c-472a-a602-b456e00920d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:06.891029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391dd84f-802c-472a-a602-b456e00920d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:06.902690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391dd84f-802c-472a-a602-b456e00920d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:06.908622 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-391dd84f-802c-472a-a602-b456e00920d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:35:06.908622 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:06.908622 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:06.908622 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:06.908622 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:06.908622 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:06.908622 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:06.908622 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:06.908622 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:06.908622 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:06.908622 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:06.908622 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:06.908622 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:06.908622 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:06.908622 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:06.908622 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:06.909783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:06.909783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:06.909783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:06.909783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:06.909783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:06.909783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:06.909783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:06.909783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:06.909783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:06.909783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:06.909783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:06.909783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:06.909783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:06.909783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:06.909783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:06.909783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:06.915596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:06.915596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:06.915596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:06.915596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:06.915596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:06.915596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:06.915596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:06.915596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:06.915596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:06.915596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:06.915596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:35:06.915596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:06.915596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 106/223] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:35:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:06.915596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1ab28-76c6-49bb-8b02-09973d11cb76 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:06.916302 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1ab28-76c6-49bb-8b02-09973d11cb76 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:06.916302 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf1ab28-76c6-49bb-8b02-09973d11cb76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:06.932898 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-fdf1ab28-76c6-49bb-8b02-09973d11cb76 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:06.934205 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-fdf1ab28-76c6-49bb-8b02-09973d11cb76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:06.935922 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9087e-3740-49f5-aa85-0631d5a7df91 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:06.975988 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f253f4-6673-414c-8568-fdf3ff02f5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:06.976786 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f253f4-6673-414c-8568-fdf3ff02f5dd None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:06.977142 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f253f4-6673-414c-8568-fdf3ff02f5dd None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:06.977508 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f253f4-6673-414c-8568-fdf3ff02f5dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:06.978294 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f253f4-6673-414c-8568-fdf3ff02f5dd None admin] RBAC: Adding query filter params (role.id=33c5141e078844bbb60127f2b26540d7, scope.project.id=21b65aad6f794b3588a867ef02e946ac, user.id=06535fb063c04013957d150e411bec79) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:06.978604 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f253f4-6673-414c-8568-fdf3ff02f5dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:06.980359 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f253f4-6673-414c-8568-fdf3ff02f5dd None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:06.987140 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 118/224] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:35:06 2022] GET /identity/v3/role_assignments?user.id=06535fb063c04013957d150e411bec79&scope.project.id=21b65aad6f794b3588a867ef02e946ac&role.id=33c5141e078844bbb60127f2b26540d7 => generated 260 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:06.988150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f253f4-6673-414c-8568-fdf3ff02f5dd None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:07.036228 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 107/225] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:06 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:07.082857 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf1ab28-76c6-49bb-8b02-09973d11cb76 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:07.133866 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:07.168644 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f253f4-6673-414c-8568-fdf3ff02f5dd None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:07.196241 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:07.236403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d81b69-878f-4931-b1cb-849838a07bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:07.237120 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d81b69-878f-4931-b1cb-849838a07bdc None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:07.237429 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d81b69-878f-4931-b1cb-849838a07bdc None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:07.237687 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d81b69-878f-4931-b1cb-849838a07bdc None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:07.238483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d81b69-878f-4931-b1cb-849838a07bdc None admin] RBAC: Adding query filter params (name=admin) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:07.238797 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d81b69-878f-4931-b1cb-849838a07bdc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:07.240238 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d81b69-878f-4931-b1cb-849838a07bdc None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:07.250493 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 119/226] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:35:06 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:07.257880 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d81b69-878f-4931-b1cb-849838a07bdc None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:07.300146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf1ab28-76c6-49bb-8b02-09973d11cb76 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:07.383330 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2142624-d08b-47cb-b3df-46a0b808bf36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:07.385190 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2142624-d08b-47cb-b3df-46a0b808bf36 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:07.386796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2142624-d08b-47cb-b3df-46a0b808bf36 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:07.388415 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2142624-d08b-47cb-b3df-46a0b808bf36 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:07.411642 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2142624-d08b-47cb-b3df-46a0b808bf36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:07.426092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-a2142624-d08b-47cb-b3df-46a0b808bf36 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:35:07.426092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:07.426092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:07.426092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:07.426092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:07.426092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:07.426092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:07.426092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:07.426092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:07.426092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:07.426092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:07.426092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:07.426092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:07.426092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:07.426092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:07.426092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:07.427064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:07.427064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:07.427064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:07.427064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:07.427064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:07.427064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:07.427064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:07.427064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:07.427064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:07.427064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:07.427064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:07.427064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:07.427064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:07.427064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:07.427064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:07.427064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:07.427885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:07.427885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:07.427885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:07.427885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:07.427885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:07.427885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:07.427885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:07.427885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:07.427885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:07.427885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:07.427885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:35:07.427885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:07.430850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 108/227] 10.222.0.115 () {56 vars in 1119 bytes} [Tue Aug 2 10:35:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 349 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:35:07.436541 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2142624-d08b-47cb-b3df-46a0b808bf36 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:07.488811 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d81b69-878f-4931-b1cb-849838a07bdc None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:07.566263 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:07.583184 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae15ceee-c2cf-4f24-95a8-2a9dbbc589f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:07.583905 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15ceee-c2cf-4f24-95a8-2a9dbbc589f3 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:07.584189 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15ceee-c2cf-4f24-95a8-2a9dbbc589f3 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:07.584459 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae15ceee-c2cf-4f24-95a8-2a9dbbc589f3 None admin] PATH_INFO: `/v3/users/544ca40877f949e6a646197c40282751` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:07.584652 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2142624-d08b-47cb-b3df-46a0b808bf36 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:07.592195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae15ceee-c2cf-4f24-95a8-2a9dbbc589f3 None admin] RBAC: Authorizing `identity:get_user(user_id=544ca40877f949e6a646197c40282751)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:07.593927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae15ceee-c2cf-4f24-95a8-2a9dbbc589f3 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:07.596546 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 120/228] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:35:07 2022] GET /identity/v3/users/544ca40877f949e6a646197c40282751 => generated 284 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:07.619970 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae15ceee-c2cf-4f24-95a8-2a9dbbc589f3 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:07.627802 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59dc659-b168-40b9-9709-c009958f0cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:07.628431 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dc659-b168-40b9-9709-c009958f0cfd None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:07.628714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dc659-b168-40b9-9709-c009958f0cfd None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:07.628994 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59dc659-b168-40b9-9709-c009958f0cfd None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:07.629722 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59dc659-b168-40b9-9709-c009958f0cfd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:07.630026 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59dc659-b168-40b9-9709-c009958f0cfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:07.631751 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59dc659-b168-40b9-9709-c009958f0cfd None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:07.641595 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 109/229] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:35:07 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:35:07.646711 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59dc659-b168-40b9-9709-c009958f0cfd None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:07.729400 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:07.800100 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae15ceee-c2cf-4f24-95a8-2a9dbbc589f3 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:07.839354 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59dc659-b168-40b9-9709-c009958f0cfd None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:07.846002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c0f6c-639c-4ac7-8982-e8866eb1cb97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:07.846936 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c0f6c-639c-4ac7-8982-e8866eb1cb97 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:07.847236 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c0f6c-639c-4ac7-8982-e8866eb1cb97 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:07.847506 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c0f6c-639c-4ac7-8982-e8866eb1cb97 None admin] PATH_INFO: `/v3/projects/21b65aad6f794b3588a867ef02e946ac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:07.848695 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c0f6c-639c-4ac7-8982-e8866eb1cb97 None admin] RBAC: Authorizing `identity:get_project(project_id=21b65aad6f794b3588a867ef02e946ac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:07.850188 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c0f6c-639c-4ac7-8982-e8866eb1cb97 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:07.856593 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 121/230] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:35:07 2022] GET /identity/v3/projects/21b65aad6f794b3588a867ef02e946ac => generated 303 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:07.858395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c0f6c-639c-4ac7-8982-e8866eb1cb97 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:07.922160 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ea0e0-9a16-4203-8c51-bda7d0fe4b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:07.922729 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ea0e0-9a16-4203-8c51-bda7d0fe4b05 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:07.923005 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ea0e0-9a16-4203-8c51-bda7d0fe4b05 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:07.924310 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5ea0e0-9a16-4203-8c51-bda7d0fe4b05 None admin] PATH_INFO: `/v3/users/544ca40877f949e6a646197c40282751` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:07.931035 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5ea0e0-9a16-4203-8c51-bda7d0fe4b05 None admin] RBAC: Authorizing `identity:get_user(user_id=544ca40877f949e6a646197c40282751)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:07.932505 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5ea0e0-9a16-4203-8c51-bda7d0fe4b05 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:07.934168 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 110/231] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:35:07 2022] GET /identity/v3/users/544ca40877f949e6a646197c40282751 => generated 284 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:07.940434 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ea0e0-9a16-4203-8c51-bda7d0fe4b05 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:07.977348 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:08.011706 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c0f6c-639c-4ac7-8982-e8866eb1cb97 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:08.029983 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:08.056145 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec0118-99b4-4d3a-94f8-6c368461acf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:08.056826 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec0118-99b4-4d3a-94f8-6c368461acf8 None admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:08.057496 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec0118-99b4-4d3a-94f8-6c368461acf8 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:08.057833 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdec0118-99b4-4d3a-94f8-6c368461acf8 None admin] PATH_INFO: `/v3/projects/21b65aad6f794b3588a867ef02e946ac/users/544ca40877f949e6a646197c40282751/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:08.059885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec0118-99b4-4d3a-94f8-6c368461acf8 None admin] RBAC: Authorizing `identity:create_grant(project_id=21b65aad6f794b3588a867ef02e946ac, user_id=544ca40877f949e6a646197c40282751, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:08.062811 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdec0118-99b4-4d3a-94f8-6c368461acf8 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:08.118166 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5ea0e0-9a16-4203-8c51-bda7d0fe4b05 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:08.162059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 122/232] 10.222.0.115 () {58 vars in 1444 bytes} [Tue Aug 2 10:35:07 2022] PUT /identity/v3/projects/21b65aad6f794b3588a867ef02e946ac/users/544ca40877f949e6a646197c40282751/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:35:08.162701 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77726d-ec9e-4a6b-abc7-7c3868c83edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:08.163455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77726d-ec9e-4a6b-abc7-7c3868c83edf None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:08.163731 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77726d-ec9e-4a6b-abc7-7c3868c83edf None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:08.163997 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77726d-ec9e-4a6b-abc7-7c3868c83edf None admin] PATH_INFO: `/v3/projects/21b65aad6f794b3588a867ef02e946ac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:08.165217 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77726d-ec9e-4a6b-abc7-7c3868c83edf None admin] RBAC: Authorizing `identity:get_project(project_id=21b65aad6f794b3588a867ef02e946ac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:08.166823 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77726d-ec9e-4a6b-abc7-7c3868c83edf None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:08.168803 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 111/233] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:35:07 2022] GET /identity/v3/projects/21b65aad6f794b3588a867ef02e946ac => generated 303 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:08.175929 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec0118-99b4-4d3a-94f8-6c368461acf8 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:08.327487 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdec0118-99b4-4d3a-94f8-6c368461acf8 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:08.373411 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d797ca53-a705-49f5-9187-c896e6a7bca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:08.374097 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797ca53-a705-49f5-9187-c896e6a7bca4 None admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:08.374380 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797ca53-a705-49f5-9187-c896e6a7bca4 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:08.374661 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d797ca53-a705-49f5-9187-c896e6a7bca4 None admin] PATH_INFO: `/v3/projects/21b65aad6f794b3588a867ef02e946ac/users/544ca40877f949e6a646197c40282751/roles/a4213d713b6f49d78d7bbb9484c9312f` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:08.377959 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d797ca53-a705-49f5-9187-c896e6a7bca4 None admin] RBAC: Authorizing `identity:create_grant(project_id=21b65aad6f794b3588a867ef02e946ac, user_id=544ca40877f949e6a646197c40282751, role_id=a4213d713b6f49d78d7bbb9484c9312f)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:08.379457 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d797ca53-a705-49f5-9187-c896e6a7bca4 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:08.446351 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 123/234] 10.222.0.115 () {58 vars in 1444 bytes} [Tue Aug 2 10:35:08 2022] PUT /identity/v3/projects/21b65aad6f794b3588a867ef02e946ac/users/544ca40877f949e6a646197c40282751/roles/a4213d713b6f49d78d7bbb9484c9312f => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:35:08.781497 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c39eaf3-fb95-472b-b600-5adcb2d6f9d1 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:08.782044 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c39eaf3-fb95-472b-b600-5adcb2d6f9d1 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:08.782044 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c39eaf3-fb95-472b-b600-5adcb2d6f9d1 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:08.784516 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 112/235] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:08 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:08.790419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7bfd2-9fba-483a-a21e-07b6c90aeb11 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:08.791344 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7bfd2-9fba-483a-a21e-07b6c90aeb11 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:08.791819 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb7bfd2-9fba-483a-a21e-07b6c90aeb11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:08.813572 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-8fb7bfd2-9fba-483a-a21e-07b6c90aeb11 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:08.815715 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-8fb7bfd2-9fba-483a-a21e-07b6c90aeb11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:08.858126 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:08.900640 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d387d-8081-4d54-91b0-3f8f969e1611 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:08.900981 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d387d-8081-4d54-91b0-3f8f969e1611 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:08.901190 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566d387d-8081-4d54-91b0-3f8f969e1611 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:08.902938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 113/236] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:08 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:08.914942 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36576913-f6f6-4c0c-9690-83d5917ecc0d None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:08.915246 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36576913-f6f6-4c0c-9690-83d5917ecc0d None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:08.915525 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36576913-f6f6-4c0c-9690-83d5917ecc0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:08.938700 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 124/237] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:08 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:08.939721 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-36576913-f6f6-4c0c-9690-83d5917ecc0d None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:08.941009 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-36576913-f6f6-4c0c-9690-83d5917ecc0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:08.968159 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb7bfd2-9fba-483a-a21e-07b6c90aeb11 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:09.074497 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 114/238] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:08 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:35:09.101640 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36576913-f6f6-4c0c-9690-83d5917ecc0d None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:09.131477 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:09.169085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb7bfd2-9fba-483a-a21e-07b6c90aeb11 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:09.232146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36576913-f6f6-4c0c-9690-83d5917ecc0d None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:09.274778 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05431b46-f338-4cee-88c3-75a1f9d0038b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:09.275471 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05431b46-f338-4cee-88c3-75a1f9d0038b None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:09.275908 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05431b46-f338-4cee-88c3-75a1f9d0038b None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:09.276451 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05431b46-f338-4cee-88c3-75a1f9d0038b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:09.292771 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05431b46-f338-4cee-88c3-75a1f9d0038b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:09.294341 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f916633-07d7-40f8-8192-b25fd0e21a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:09.294989 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f916633-07d7-40f8-8192-b25fd0e21a69 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:09.295410 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f916633-07d7-40f8-8192-b25fd0e21a69 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:09.295679 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f916633-07d7-40f8-8192-b25fd0e21a69 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:09.297198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f916633-07d7-40f8-8192-b25fd0e21a69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:09.299210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f916633-07d7-40f8-8192-b25fd0e21a69 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:09.301302 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 125/239] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:08 2022] GET /identity/v3/domains/default => generated 200 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:09.305728 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-05431b46-f338-4cee-88c3-75a1f9d0038b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:35:09.305728 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:09.305728 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:09.305728 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:09.305728 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:09.305728 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:09.305728 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:09.305728 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:09.305728 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:09.305728 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:09.305728 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:09.305728 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:09.305728 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:09.305728 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:09.305728 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:09.305728 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:09.306651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:09.306651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:09.306651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:09.306651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:09.306651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:09.306651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:09.306651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:09.306651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:09.306651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:09.306651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:09.306651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:09.306651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:09.306651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:09.306651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:09.306651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:09.306651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:09.307508 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:09.307508 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:09.307508 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:09.307508 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:09.307508 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:09.307508 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:09.307508 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:09.307508 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:09.307508 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:09.307508 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:09.307508 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:35:09.307508 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:09.307508 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 115/240] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:35:09 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:35:09.308186 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f916633-07d7-40f8-8192-b25fd0e21a69 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:09.317343 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05431b46-f338-4cee-88c3-75a1f9d0038b None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:09.401669 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:09.436369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:09.514980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f916633-07d7-40f8-8192-b25fd0e21a69 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:09.527589 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05431b46-f338-4cee-88c3-75a1f9d0038b None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:09.582643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efab04b-20a4-4b15-b254-7ea1a26b7aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:09.583256 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336e377-75b3-462a-96dc-604adcf14731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:09.583376 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efab04b-20a4-4b15-b254-7ea1a26b7aaf None admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:09.583604 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efab04b-20a4-4b15-b254-7ea1a26b7aaf None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:09.583868 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efab04b-20a4-4b15-b254-7ea1a26b7aaf None admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:09.583977 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336e377-75b3-462a-96dc-604adcf14731 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:09.584191 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336e377-75b3-462a-96dc-604adcf14731 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:09.584471 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336e377-75b3-462a-96dc-604adcf14731 None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:09.584657 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efab04b-20a4-4b15-b254-7ea1a26b7aaf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:09.585190 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8336e377-75b3-462a-96dc-604adcf14731 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:09.585495 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8336e377-75b3-462a-96dc-604adcf14731 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:09.586110 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efab04b-20a4-4b15-b254-7ea1a26b7aaf None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:09.586943 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8336e377-75b3-462a-96dc-604adcf14731 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:09.597545 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 116/241] 10.222.0.115 () {56 vars in 1106 bytes} [Tue Aug 2 10:35:09 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:09.598906 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07e24d-bd8e-4886-8042-576bb076e065 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:09.599222 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07e24d-bd8e-4886-8042-576bb076e065 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:09.599503 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a07e24d-bd8e-4886-8042-576bb076e065 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:09.600743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 117/242] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:09 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:09.604149 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a07e24d-bd8e-4886-8042-576bb076e065 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:09.683231 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 126/243] 10.222.0.115 () {60 vars in 1137 bytes} [Tue Aug 2 10:35:09 2022] POST /identity/v3/users => generated 294 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:35:09.684677 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e13c34-8dc9-410e-8c87-8cc5cbc4f4ba None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:09.684974 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e13c34-8dc9-410e-8c87-8cc5cbc4f4ba None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:09.685256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e13c34-8dc9-410e-8c87-8cc5cbc4f4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:09.729090 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-c9e13c34-8dc9-410e-8c87-8cc5cbc4f4ba None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:09.731243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-c9e13c34-8dc9-410e-8c87-8cc5cbc4f4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:09.759380 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a07e24d-bd8e-4886-8042-576bb076e065 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:09.839183 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ddcbf-3f7e-4f6d-95b7-3e392ff973d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:09.839962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ddcbf-3f7e-4f6d-95b7-3e392ff973d0 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:09.840129 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ddcbf-3f7e-4f6d-95b7-3e392ff973d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:09.840393 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ddcbf-3f7e-4f6d-95b7-3e392ff973d0 None admin] PATH_INFO: `/v3/users/5e30988273d44931859a959d14bcffad` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:09.859495 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:09.874330 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5ddcbf-3f7e-4f6d-95b7-3e392ff973d0 None admin] RBAC: Authorizing `identity:get_user(user_id=5e30988273d44931859a959d14bcffad)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:09.875837 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5ddcbf-3f7e-4f6d-95b7-3e392ff973d0 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:09.878613 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 118/244] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:35:09 2022] GET /identity/v3/users/5e30988273d44931859a959d14bcffad => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:09.885370 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ddcbf-3f7e-4f6d-95b7-3e392ff973d0 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:09.906679 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 127/245] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:09 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:09.933435 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e13c34-8dc9-410e-8c87-8cc5cbc4f4ba None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:09.959438 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:09.999656 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ddcbf-3f7e-4f6d-95b7-3e392ff973d0 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:10.038084 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea4d8af-6e6e-42ba-899f-a2819dc58f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:10.038716 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea4d8af-6e6e-42ba-899f-a2819dc58f89 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:10.038990 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea4d8af-6e6e-42ba-899f-a2819dc58f89 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:10.039290 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea4d8af-6e6e-42ba-899f-a2819dc58f89 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:10.075119 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e13c34-8dc9-410e-8c87-8cc5cbc4f4ba None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:10.108815 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:10.119688 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea4d8af-6e6e-42ba-899f-a2819dc58f89 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:10.130024 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea4d8af-6e6e-42ba-899f-a2819dc58f89 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:10.130173 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90444f7-fcba-4622-8a75-312195bae8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:10.130653 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90444f7-fcba-4622-8a75-312195bae8f0 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:10.132385 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90444f7-fcba-4622-8a75-312195bae8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:10.132928 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90444f7-fcba-4622-8a75-312195bae8f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:10.155528 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90444f7-fcba-4622-8a75-312195bae8f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:10.161874 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-e90444f7-fcba-4622-8a75-312195bae8f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:35:10.161874 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:10.161874 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:10.161874 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:10.161874 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:10.161874 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:10.161874 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:10.161874 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:10.161874 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:10.161874 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:10.161874 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:10.161874 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:10.161874 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:10.161874 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:10.161874 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:10.161874 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:10.162815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:10.162815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:10.162815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:10.162815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:10.162815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:10.162815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:10.162815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:10.162815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:10.162815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:10.162815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:10.162815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:10.162815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:10.162815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:10.162815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:10.162815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:10.162815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:10.163659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:10.163659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:10.163659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:10.163659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:10.163659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:10.163659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:10.163659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:10.163659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:10.163659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:10.163659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:10.163659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:35:10.163659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:10.165907 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 128/246] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:35:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:35:10.172554 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90444f7-fcba-4622-8a75-312195bae8f0 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:10.242411 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 119/247] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:35:09 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:35:10.250787 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea4d8af-6e6e-42ba-899f-a2819dc58f89 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:10.296657 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:10.325990 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90444f7-fcba-4622-8a75-312195bae8f0 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:10.382309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889d7c0-bb8d-4ece-9e07-b41ee1386495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:10.382943 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889d7c0-bb8d-4ece-9e07-b41ee1386495 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:10.383238 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889d7c0-bb8d-4ece-9e07-b41ee1386495 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:10.383500 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:10.383603 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3889d7c0-bb8d-4ece-9e07-b41ee1386495 None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:10.384222 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3889d7c0-bb8d-4ece-9e07-b41ee1386495 None admin] RBAC: Adding query filter params (name=member) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:10.384598 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3889d7c0-bb8d-4ece-9e07-b41ee1386495 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:10.385996 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3889d7c0-bb8d-4ece-9e07-b41ee1386495 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:10.399317 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 129/248] 10.222.0.115 () {56 vars in 1106 bytes} [Tue Aug 2 10:35:10 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:35:10.405025 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea4d8af-6e6e-42ba-899f-a2819dc58f89 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:10.407144 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889d7c0-bb8d-4ece-9e07-b41ee1386495 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:10.453826 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b3ae8-b5e0-465a-8d25-3fb1deb03bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:10.454935 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b3ae8-b5e0-465a-8d25-3fb1deb03bb9 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:10.455529 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b3ae8-b5e0-465a-8d25-3fb1deb03bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:10.456076 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0b3ae8-b5e0-465a-8d25-3fb1deb03bb9 None admin] PATH_INFO: `/v3/projects/21b65aad6f794b3588a867ef02e946ac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:10.458049 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b3ae8-b5e0-465a-8d25-3fb1deb03bb9 None admin] RBAC: Authorizing `identity:get_project(project_id=21b65aad6f794b3588a867ef02e946ac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:10.460718 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0b3ae8-b5e0-465a-8d25-3fb1deb03bb9 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:10.463189 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 120/249] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:35:10 2022] GET /identity/v3/projects/21b65aad6f794b3588a867ef02e946ac => generated 303 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:10.470807 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b3ae8-b5e0-465a-8d25-3fb1deb03bb9 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:10.565445 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3889d7c0-bb8d-4ece-9e07-b41ee1386495 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:10.612853 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd975d37-7d2a-48de-95b1-b7915631b9c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:10.613615 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd975d37-7d2a-48de-95b1-b7915631b9c9 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:10.613819 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd975d37-7d2a-48de-95b1-b7915631b9c9 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:10.614089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd975d37-7d2a-48de-95b1-b7915631b9c9 None admin] PATH_INFO: `/v3/users/06535fb063c04013957d150e411bec79` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:10.615310 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd975d37-7d2a-48de-95b1-b7915631b9c9 None admin] RBAC: Authorizing `identity:get_user(user_id=06535fb063c04013957d150e411bec79)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:10.616771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd975d37-7d2a-48de-95b1-b7915631b9c9 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:10.618426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 130/250] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:35:10 2022] GET /identity/v3/users/06535fb063c04013957d150e411bec79 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:10.620950 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763acd0-7814-4439-a2fd-257cd9395b02 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:10.621235 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763acd0-7814-4439-a2fd-257cd9395b02 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:10.621513 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763acd0-7814-4439-a2fd-257cd9395b02 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:10.623316 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 131/251] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:10 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:10.624144 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f763acd0-7814-4439-a2fd-257cd9395b02 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:10.635823 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0b3ae8-b5e0-465a-8d25-3fb1deb03bb9 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:10.639908 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:10.690465 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e782f3-78fa-4851-98c9-d2fbb3f717c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:10.692017 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e782f3-78fa-4851-98c9-d2fbb3f717c5 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:10.692674 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e782f3-78fa-4851-98c9-d2fbb3f717c5 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:10.693233 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e782f3-78fa-4851-98c9-d2fbb3f717c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:10.694670 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e782f3-78fa-4851-98c9-d2fbb3f717c5 None admin] RBAC: Adding query filter params (role.id=1cf433b432764fbab711f54dcf7aa81e, scope.project.id=21b65aad6f794b3588a867ef02e946ac, user.id=5e30988273d44931859a959d14bcffad) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:10.695197 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e782f3-78fa-4851-98c9-d2fbb3f717c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:10.697595 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e782f3-78fa-4851-98c9-d2fbb3f717c5 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:10.711309 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 121/252] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:35:10 2022] GET /identity/v3/role_assignments?user.id=5e30988273d44931859a959d14bcffad&scope.project.id=21b65aad6f794b3588a867ef02e946ac&role.id=1cf433b432764fbab711f54dcf7aa81e => generated 260 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:10.715080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23bb8d-1c97-4e23-9b48-d773a42ee0f8 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:10.715859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23bb8d-1c97-4e23-9b48-d773a42ee0f8 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:10.716336 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23bb8d-1c97-4e23-9b48-d773a42ee0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:10.764514 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-5b23bb8d-1c97-4e23-9b48-d773a42ee0f8 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:10.766535 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-5b23bb8d-1c97-4e23-9b48-d773a42ee0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:10.810853 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:10.856237 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f763acd0-7814-4439-a2fd-257cd9395b02 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:10.887958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 122/253] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:10 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:10.917351 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326947ab-7b05-4308-8dd1-baefb86d965d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:10.918061 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326947ab-7b05-4308-8dd1-baefb86d965d None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:10.918442 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326947ab-7b05-4308-8dd1-baefb86d965d None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:10.918680 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326947ab-7b05-4308-8dd1-baefb86d965d None admin] PATH_INFO: `/v3/projects/21b65aad6f794b3588a867ef02e946ac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:10.920074 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326947ab-7b05-4308-8dd1-baefb86d965d None admin] RBAC: Authorizing `identity:get_project(project_id=21b65aad6f794b3588a867ef02e946ac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:10.921868 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326947ab-7b05-4308-8dd1-baefb86d965d None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:10.924515 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 132/254] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:35:10 2022] GET /identity/v3/projects/21b65aad6f794b3588a867ef02e946ac => generated 303 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:10.928927 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23bb8d-1c97-4e23-9b48-d773a42ee0f8 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:10.941023 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326947ab-7b05-4308-8dd1-baefb86d965d None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:11.059894 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:11.089293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326947ab-7b05-4308-8dd1-baefb86d965d None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:11.136465 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eb8fb5-6e63-436b-a328-be32ce999ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:11.137443 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb8fb5-6e63-436b-a328-be32ce999ad2 None admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:11.137868 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb8fb5-6e63-436b-a328-be32ce999ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:11.138341 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eb8fb5-6e63-436b-a328-be32ce999ad2 None admin] PATH_INFO: `/v3/projects/21b65aad6f794b3588a867ef02e946ac/users/06535fb063c04013957d150e411bec79/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:11.142408 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb8fb5-6e63-436b-a328-be32ce999ad2 None admin] RBAC: Authorizing `identity:create_grant(project_id=21b65aad6f794b3588a867ef02e946ac, user_id=06535fb063c04013957d150e411bec79, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:11.144806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eb8fb5-6e63-436b-a328-be32ce999ad2 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:11.155386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23bb8d-1c97-4e23-9b48-d773a42ee0f8 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:11.196899 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40016a2-2719-4d70-8809-d55ea5571e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:11.197873 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40016a2-2719-4d70-8809-d55ea5571e41 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:11.198289 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40016a2-2719-4d70-8809-d55ea5571e41 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:11.198748 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40016a2-2719-4d70-8809-d55ea5571e41 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:11.219911 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 133/255] 10.222.0.115 () {58 vars in 1444 bytes} [Tue Aug 2 10:35:10 2022] PUT /identity/v3/projects/21b65aad6f794b3588a867ef02e946ac/users/06535fb063c04013957d150e411bec79/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:35:11.228437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40016a2-2719-4d70-8809-d55ea5571e41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:11.235038 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e7811-bfe7-4cc8-9ac1-564c43c535a4 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:11.236162 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e7811-bfe7-4cc8-9ac1-564c43c535a4 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:11.236879 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62e7811-bfe7-4cc8-9ac1-564c43c535a4 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:11.241207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 134/256] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:11 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:35:11.243092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-e40016a2-2719-4d70-8809-d55ea5571e41 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:35:11.243092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:11.243092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:11.243092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:11.243092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:11.243092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:11.243092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:11.243092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:11.243092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:11.243092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:11.243092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:11.243092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:11.243092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:11.243092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:11.243092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:11.243092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:11.244649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:11.244649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:11.244649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:11.244649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:11.244649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:11.244649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:11.244649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:11.244649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:11.244649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:11.244649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:11.244649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:11.244649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:11.244649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:11.244649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:11.244649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:11.244649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:11.245676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:11.245676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:11.245676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:11.245676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:11.245676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:11.245676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:11.245676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:11.245676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:11.245676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:11.245676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:11.245676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:35:11.245676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:11.250238 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51260d94-1250-4a6d-a712-10bc331fca93 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:11.250714 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51260d94-1250-4a6d-a712-10bc331fca93 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:11.251215 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51260d94-1250-4a6d-a712-10bc331fca93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:11.262657 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 123/257] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:35:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:11.266864 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40016a2-2719-4d70-8809-d55ea5571e41 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:11.296354 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-51260d94-1250-4a6d-a712-10bc331fca93 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:11.298723 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-51260d94-1250-4a6d-a712-10bc331fca93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:11.309653 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:11.477429 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 135/258] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:11 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:35:11.486596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40016a2-2719-4d70-8809-d55ea5571e41 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:11.530317 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51260d94-1250-4a6d-a712-10bc331fca93 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:11.581039 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b548f2a-1070-4fe5-81fa-4f56f4fc16c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:11.582073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b548f2a-1070-4fe5-81fa-4f56f4fc16c6 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:11.582360 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b548f2a-1070-4fe5-81fa-4f56f4fc16c6 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:11.582631 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b548f2a-1070-4fe5-81fa-4f56f4fc16c6 None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:11.583381 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b548f2a-1070-4fe5-81fa-4f56f4fc16c6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:11.583683 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b548f2a-1070-4fe5-81fa-4f56f4fc16c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:11.585177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b548f2a-1070-4fe5-81fa-4f56f4fc16c6 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:11.612477 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 124/259] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:35:11 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:35:11.618945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b548f2a-1070-4fe5-81fa-4f56f4fc16c6 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:11.731300 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:11.761125 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:11.789355 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b548f2a-1070-4fe5-81fa-4f56f4fc16c6 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:11.830492 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51260d94-1250-4a6d-a712-10bc331fca93 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:11.844788 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d7ec8-a2b7-4d3f-8ba6-70d6a36725dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:11.845983 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d7ec8-a2b7-4d3f-8ba6-70d6a36725dc None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:11.846264 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d7ec8-a2b7-4d3f-8ba6-70d6a36725dc None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:11.846536 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d7ec8-a2b7-4d3f-8ba6-70d6a36725dc None admin] PATH_INFO: `/v3/users/544ca40877f949e6a646197c40282751` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:11.848115 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809d7ec8-a2b7-4d3f-8ba6-70d6a36725dc None admin] RBAC: Authorizing `identity:get_user(user_id=544ca40877f949e6a646197c40282751)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:11.852531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809d7ec8-a2b7-4d3f-8ba6-70d6a36725dc None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:11.854308 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 125/260] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:35:11 2022] GET /identity/v3/users/544ca40877f949e6a646197c40282751 => generated 284 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:35:11.860131 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d7ec8-a2b7-4d3f-8ba6-70d6a36725dc None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:11.924294 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b0b2f9-ba46-4823-a269-67954f3957ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:11.924947 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0b2f9-ba46-4823-a269-67954f3957ba None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:11.927389 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0b2f9-ba46-4823-a269-67954f3957ba None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:11.927724 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0b2f9-ba46-4823-a269-67954f3957ba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:11.967084 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b0b2f9-ba46-4823-a269-67954f3957ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:11.985152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-23b0b2f9-ba46-4823-a269-67954f3957ba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:35:11.985152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:11.985152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:11.985152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:11.985152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:11.985152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:11.985152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:11.985152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:11.985152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:11.985152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:11.985152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:11.985152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:11.985152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:11.985152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:11.985152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:11.985152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:11.988158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:11.988158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:11.988158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:11.988158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:11.988158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:11.988158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:11.988158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:11.988158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:11.988158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:11.988158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:11.988158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:11.988158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:11.988158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:11.988158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:11.988158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:11.988158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:11.990171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:11.990171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:11.990171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:11.990171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:11.990171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:11.990171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:11.990171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:11.990171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:11.990171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:11.990171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:11.990171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:35:11.990171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:11.990171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 136/261] 10.222.0.115 () {56 vars in 1119 bytes} [Tue Aug 2 10:35:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 459 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:35:12.000703 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b0b2f9-ba46-4823-a269-67954f3957ba None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:12.082535 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d7ec8-a2b7-4d3f-8ba6-70d6a36725dc None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:12.127965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a9d272-a474-4f55-95e3-e36a50395b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:12.128611 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9d272-a474-4f55-95e3-e36a50395b79 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:12.128896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9d272-a474-4f55-95e3-e36a50395b79 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:12.129262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a9d272-a474-4f55-95e3-e36a50395b79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:12.150721 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:12.244195 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a9d272-a474-4f55-95e3-e36a50395b79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:12.246142 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a9d272-a474-4f55-95e3-e36a50395b79 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:12.319314 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b0b2f9-ba46-4823-a269-67954f3957ba None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:12.363794 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:12.382015 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94dc17-5bc8-4c23-b6d9-75902bae5ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:12.382652 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94dc17-5bc8-4c23-b6d9-75902bae5ff8 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:12.382937 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94dc17-5bc8-4c23-b6d9-75902bae5ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:12.383364 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94dc17-5bc8-4c23-b6d9-75902bae5ff8 None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:12.384109 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e94dc17-5bc8-4c23-b6d9-75902bae5ff8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:12.384418 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e94dc17-5bc8-4c23-b6d9-75902bae5ff8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:12.386099 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e94dc17-5bc8-4c23-b6d9-75902bae5ff8 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:12.405683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 137/262] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:35:11 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:12.411874 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94dc17-5bc8-4c23-b6d9-75902bae5ff8 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:12.515750 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 126/263] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:35:11 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 656 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:35:12.517280 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a9d272-a474-4f55-95e3-e36a50395b79 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:12.574567 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:12.636068 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a9d272-a474-4f55-95e3-e36a50395b79 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:12.665452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94dc17-5bc8-4c23-b6d9-75902bae5ff8 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:12.685096 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e89f1-fa2d-412d-9742-d97eea9fd99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:12.686328 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e89f1-fa2d-412d-9742-d97eea9fd99a None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:12.686730 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e89f1-fa2d-412d-9742-d97eea9fd99a None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:12.687345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e89f1-fa2d-412d-9742-d97eea9fd99a None admin] PATH_INFO: `/v3/projects/21b65aad6f794b3588a867ef02e946ac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:12.688821 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e89f1-fa2d-412d-9742-d97eea9fd99a None admin] RBAC: Authorizing `identity:get_project(project_id=21b65aad6f794b3588a867ef02e946ac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:12.690316 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e89f1-fa2d-412d-9742-d97eea9fd99a None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:12.692847 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 127/264] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:35:12 2022] GET /identity/v3/projects/21b65aad6f794b3588a867ef02e946ac => generated 303 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:12.699359 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e89f1-fa2d-412d-9742-d97eea9fd99a None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:12.777558 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf89426-a567-4f4d-827d-d7b6b2386d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:12.778223 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf89426-a567-4f4d-827d-d7b6b2386d08 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:12.778503 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf89426-a567-4f4d-827d-d7b6b2386d08 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:12.778775 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf89426-a567-4f4d-827d-d7b6b2386d08 None admin] PATH_INFO: `/v3/users/544ca40877f949e6a646197c40282751` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:12.780718 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf89426-a567-4f4d-827d-d7b6b2386d08 None admin] RBAC: Authorizing `identity:get_user(user_id=544ca40877f949e6a646197c40282751)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:12.784796 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf89426-a567-4f4d-827d-d7b6b2386d08 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:12.786669 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 138/265] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:35:12 2022] GET /identity/v3/users/544ca40877f949e6a646197c40282751 => generated 284 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:12.792682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf89426-a567-4f4d-827d-d7b6b2386d08 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:12.853441 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e89f1-fa2d-412d-9742-d97eea9fd99a None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:12.861603 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:12.889991 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:12.933808 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fbae0f-39e9-4540-9143-61f3309f76de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:12.935337 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fbae0f-39e9-4540-9143-61f3309f76de None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:12.936207 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fbae0f-39e9-4540-9143-61f3309f76de None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:12.936938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fbae0f-39e9-4540-9143-61f3309f76de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:12.939200 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf89426-a567-4f4d-827d-d7b6b2386d08 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:12.939884 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fbae0f-39e9-4540-9143-61f3309f76de None admin] RBAC: Adding query filter params (role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac, scope.project.id=21b65aad6f794b3588a867ef02e946ac, user.id=544ca40877f949e6a646197c40282751) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:12.939884 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fbae0f-39e9-4540-9143-61f3309f76de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:12.947791 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fbae0f-39e9-4540-9143-61f3309f76de None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:12.962646 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 128/266] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:35:12 2022] GET /identity/v3/role_assignments?user.id=544ca40877f949e6a646197c40282751&scope.project.id=21b65aad6f794b3588a867ef02e946ac&role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 610 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:12.989055 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fe757d-1c4a-4717-bf17-99b476b5fe5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:12.989694 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe757d-1c4a-4717-bf17-99b476b5fe5b None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:12.989976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe757d-1c4a-4717-bf17-99b476b5fe5b None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:12.990253 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fe757d-1c4a-4717-bf17-99b476b5fe5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:13.086830 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe757d-1c4a-4717-bf17-99b476b5fe5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:13.092492 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fe757d-1c4a-4717-bf17-99b476b5fe5b None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:13.163438 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:13.215241 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 139/267] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:35:12 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:35:13.221250 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fbae0f-39e9-4540-9143-61f3309f76de None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:13.355182 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6a4ab-7886-4b15-b32b-d4032fd6f732 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:13.355474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6a4ab-7886-4b15-b32b-d4032fd6f732 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:13.355657 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6a4ab-7886-4b15-b32b-d4032fd6f732 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:13.357166 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 140/268] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:13.363295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065225c-9a36-439c-9171-c6d921e1b35c None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:13.363582 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065225c-9a36-439c-9171-c6d921e1b35c None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:13.363863 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a065225c-9a36-439c-9171-c6d921e1b35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:13.373614 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fbae0f-39e9-4540-9143-61f3309f76de None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:13.404903 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-a065225c-9a36-439c-9171-c6d921e1b35c None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:13.406125 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-a065225c-9a36-439c-9171-c6d921e1b35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:13.452081 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2d929-e912-4074-a8c2-bf3ac76a45e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:13.452743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2d929-e912-4074-a8c2-bf3ac76a45e4 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:13.453022 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2d929-e912-4074-a8c2-bf3ac76a45e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:13.453296 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf2d929-e912-4074-a8c2-bf3ac76a45e4 None admin] PATH_INFO: `/v3/projects/21b65aad6f794b3588a867ef02e946ac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:13.456302 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf2d929-e912-4074-a8c2-bf3ac76a45e4 None admin] RBAC: Authorizing `identity:get_project(project_id=21b65aad6f794b3588a867ef02e946ac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:13.458044 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf2d929-e912-4074-a8c2-bf3ac76a45e4 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:13.460370 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 129/269] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:35:13 2022] GET /identity/v3/projects/21b65aad6f794b3588a867ef02e946ac => generated 303 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:13.470842 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2d929-e912-4074-a8c2-bf3ac76a45e4 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:13.576059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 141/270] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:13 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:13.577862 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:13.646454 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065225c-9a36-439c-9171-c6d921e1b35c None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:13.656926 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf2d929-e912-4074-a8c2-bf3ac76a45e4 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:13.712545 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b51437d-64d1-4318-8784-df904c7fd7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:13.713184 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51437d-64d1-4318-8784-df904c7fd7b5 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:13.713466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51437d-64d1-4318-8784-df904c7fd7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:13.713743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b51437d-64d1-4318-8784-df904c7fd7b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:13.714553 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b51437d-64d1-4318-8784-df904c7fd7b5 None admin] RBAC: Adding query filter params (role.id=a4213d713b6f49d78d7bbb9484c9312f, scope.project.id=21b65aad6f794b3588a867ef02e946ac, user.id=544ca40877f949e6a646197c40282751) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:13.714868 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b51437d-64d1-4318-8784-df904c7fd7b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:13.716615 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b51437d-64d1-4318-8784-df904c7fd7b5 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:13.733722 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 130/271] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:35:13 2022] GET /identity/v3/role_assignments?user.id=544ca40877f949e6a646197c40282751&scope.project.id=21b65aad6f794b3588a867ef02e946ac&role.id=a4213d713b6f49d78d7bbb9484c9312f => generated 610 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:13.756009 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a51fdf-4adb-4e1d-9572-8b9c0f945ac5 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:13.756724 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a51fdf-4adb-4e1d-9572-8b9c0f945ac5 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:13.757421 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a51fdf-4adb-4e1d-9572-8b9c0f945ac5 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:13.759345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 131/272] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:13 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:35:13.765986 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0177d53-21d1-433e-887b-151f1c594b67 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:13.766733 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0177d53-21d1-433e-887b-151f1c594b67 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:13.767947 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0177d53-21d1-433e-887b-151f1c594b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:13.789016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-c0177d53-21d1-433e-887b-151f1c594b67 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:13.790662 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-c0177d53-21d1-433e-887b-151f1c594b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:13.888285 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a065225c-9a36-439c-9171-c6d921e1b35c None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:13.914045 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 132/273] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:13 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:35:13.919414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:13.935363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc549f8-b6cd-4d14-a615-f8bf334d93c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:13.936015 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc549f8-b6cd-4d14-a615-f8bf334d93c1 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:13.936297 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc549f8-b6cd-4d14-a615-f8bf334d93c1 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:13.936572 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc549f8-b6cd-4d14-a615-f8bf334d93c1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:13.941715 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0177d53-21d1-433e-887b-151f1c594b67 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:13.960224 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc549f8-b6cd-4d14-a615-f8bf334d93c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:13.965451 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-fdc549f8-b6cd-4d14-a615-f8bf334d93c1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:35:13.965451 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:13.965451 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:13.965451 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:13.965451 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:13.965451 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:13.965451 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:13.965451 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:13.965451 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:13.965451 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:13.965451 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:13.965451 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:13.965451 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:13.965451 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:13.965451 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:13.965451 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:13.966765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:13.966765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:13.966765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:13.966765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:13.966765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:13.966765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:13.966765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:13.966765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:13.966765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:13.966765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:13.966765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:13.966765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:13.966765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:13.966765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:13.966765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:13.966765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:13.967730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:13.967730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:13.967730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:13.967730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:13.967730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:13.967730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:13.967730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:13.967730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:13.967730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:13.967730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:13.967730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:35:13.967730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:13.967730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 142/274] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:35:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:35:13.968401 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4861b02-d7b2-412e-81ef-5a5fe861d400 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:13.968477 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4861b02-d7b2-412e-81ef-5a5fe861d400 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:13.973707 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4861b02-d7b2-412e-81ef-5a5fe861d400 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:13.973707 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 143/275] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:13.976575 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed96dc6e-28b5-49de-8c99-90d04aa7353d None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:13.976858 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed96dc6e-28b5-49de-8c99-90d04aa7353d None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:13.977693 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed96dc6e-28b5-49de-8c99-90d04aa7353d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:13.997089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-ed96dc6e-28b5-49de-8c99-90d04aa7353d None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:13.998662 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-ed96dc6e-28b5-49de-8c99-90d04aa7353d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:14.009039 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:14.090245 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 144/276] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:13 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:14.091627 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed96dc6e-28b5-49de-8c99-90d04aa7353d None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:14.103478 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0177d53-21d1-433e-887b-151f1c594b67 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:14.156031 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c994269c-8cfc-4045-93a3-e8f570b196ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:14.157092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c994269c-8cfc-4045-93a3-e8f570b196ee None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:14.158225 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c994269c-8cfc-4045-93a3-e8f570b196ee None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:14.158374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c994269c-8cfc-4045-93a3-e8f570b196ee None admin] PATH_INFO: `/v3/domains/default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:14.159674 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c994269c-8cfc-4045-93a3-e8f570b196ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:14.161142 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c994269c-8cfc-4045-93a3-e8f570b196ee None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:14.162941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 133/277] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:13 2022] GET /identity/v3/domains/default => generated 200 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:14.163574 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c994269c-8cfc-4045-93a3-e8f570b196ee None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:14.192637 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:14.218404 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed96dc6e-28b5-49de-8c99-90d04aa7353d None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:14.255571 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf8e03-e1f4-43f3-a430-e36fec1acb81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:14.256215 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf8e03-e1f4-43f3-a430-e36fec1acb81 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:14.256494 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf8e03-e1f4-43f3-a430-e36fec1acb81 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:14.256774 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cf8e03-e1f4-43f3-a430-e36fec1acb81 None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:14.257502 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cf8e03-e1f4-43f3-a430-e36fec1acb81 None admin] RBAC: Adding query filter params (name=member) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:14.257816 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cf8e03-e1f4-43f3-a430-e36fec1acb81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:14.259237 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cf8e03-e1f4-43f3-a430-e36fec1acb81 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:14.272963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 145/278] 10.222.0.115 () {56 vars in 1106 bytes} [Tue Aug 2 10:35:14 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:14.273711 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf8e03-e1f4-43f3-a430-e36fec1acb81 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:14.285360 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c994269c-8cfc-4045-93a3-e8f570b196ee None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:14.294431 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:14.328224 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d696ec-ff99-40a5-b941-6ad45c1be2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:14.328863 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d696ec-ff99-40a5-b941-6ad45c1be2f0 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:14.329172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d696ec-ff99-40a5-b941-6ad45c1be2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:14.329464 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d696ec-ff99-40a5-b941-6ad45c1be2f0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:14.345453 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d696ec-ff99-40a5-b941-6ad45c1be2f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:14.350922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-64d696ec-ff99-40a5-b941-6ad45c1be2f0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:35:14.350922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:14.350922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:14.350922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:14.350922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:14.350922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:14.350922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:14.350922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:14.350922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:14.350922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:14.350922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:14.350922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:14.350922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:14.350922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:14.350922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:14.350922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:14.351911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:14.351911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:14.351911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:14.351911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:14.351911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:14.351911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:14.351911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:14.351911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:14.351911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:14.351911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:14.351911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:14.351911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:14.351911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:14.351911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:14.351911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:14.351911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:14.352711 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:14.352711 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:14.352711 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:14.352711 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:14.352711 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:14.352711 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:14.352711 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:14.352711 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:14.352711 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:14.352711 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:14.352711 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:35:14.352711 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:14.352711 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 134/279] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:35:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:14.353373 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d696ec-ff99-40a5-b941-6ad45c1be2f0 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:14.398181 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cf8e03-e1f4-43f3-a430-e36fec1acb81 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:14.441697 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d5ad6-5a51-4dcf-adcf-a1631dba51cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:14.442323 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d5ad6-5a51-4dcf-adcf-a1631dba51cc None admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:14.442603 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d5ad6-5a51-4dcf-adcf-a1631dba51cc None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:14.442929 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0d5ad6-5a51-4dcf-adcf-a1631dba51cc None admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:14.444108 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d5ad6-5a51-4dcf-adcf-a1631dba51cc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:14.445921 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0d5ad6-5a51-4dcf-adcf-a1631dba51cc None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:14.462532 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:14.476907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d696ec-ff99-40a5-b941-6ad45c1be2f0 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:14.514033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a8aea-26d6-4eb1-9889-f1022ec116f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:14.514667 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a8aea-26d6-4eb1-9889-f1022ec116f7 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:14.514944 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a8aea-26d6-4eb1-9889-f1022ec116f7 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:14.515248 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161a8aea-26d6-4eb1-9889-f1022ec116f7 None admin] PATH_INFO: `/v3/users/35c42cdc208f40c2bfbd0f414e36bd95` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:14.527875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 146/280] 10.222.0.115 () {60 vars in 1137 bytes} [Tue Aug 2 10:35:14 2022] POST /identity/v3/users => generated 294 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:35:14.529880 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d5ad6-5a51-4dcf-adcf-a1631dba51cc None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:14.541338 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161a8aea-26d6-4eb1-9889-f1022ec116f7 None admin] RBAC: Authorizing `identity:get_user(user_id=35c42cdc208f40c2bfbd0f414e36bd95)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:14.543364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161a8aea-26d6-4eb1-9889-f1022ec116f7 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:14.545993 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 135/281] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:35:14 2022] GET /identity/v3/users/35c42cdc208f40c2bfbd0f414e36bd95 => generated 294 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:14.554599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a8aea-26d6-4eb1-9889-f1022ec116f7 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:14.591811 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:14.684229 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161a8aea-26d6-4eb1-9889-f1022ec116f7 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:14.684443 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0d5ad6-5a51-4dcf-adcf-a1631dba51cc None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:14.726976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f117cdb7-0ef9-4c7c-805e-75fff7f199b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:14.728415 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117cdb7-0ef9-4c7c-805e-75fff7f199b7 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:14.728415 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117cdb7-0ef9-4c7c-805e-75fff7f199b7 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:14.728415 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f117cdb7-0ef9-4c7c-805e-75fff7f199b7 None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:14.728809 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f117cdb7-0ef9-4c7c-805e-75fff7f199b7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:14.729114 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f117cdb7-0ef9-4c7c-805e-75fff7f199b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:14.730501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f117cdb7-0ef9-4c7c-805e-75fff7f199b7 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:14.730805 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7427c84-7058-44e1-abab-c1d69c5d90a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:14.731491 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7427c84-7058-44e1-abab-c1d69c5d90a7 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:14.731771 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7427c84-7058-44e1-abab-c1d69c5d90a7 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:14.732249 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7427c84-7058-44e1-abab-c1d69c5d90a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:14.733430 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7427c84-7058-44e1-abab-c1d69c5d90a7 None admin] RBAC: Adding query filter params (role.id=33c5141e078844bbb60127f2b26540d7, user.id=35c42cdc208f40c2bfbd0f414e36bd95, scope.system=all) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:14.733745 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7427c84-7058-44e1-abab-c1d69c5d90a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:14.736856 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7427c84-7058-44e1-abab-c1d69c5d90a7 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:14.740728 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 147/282] 10.222.0.115 () {56 vars in 1106 bytes} [Tue Aug 2 10:35:14 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:14.741893 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64449a10-0c95-49ed-af06-dd33ff295478 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:14.742268 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64449a10-0c95-49ed-af06-dd33ff295478 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:14.742543 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64449a10-0c95-49ed-af06-dd33ff295478 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:14.743940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 148/283] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:14 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:14.747784 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 136/284] 10.222.0.115 () {56 vars in 1313 bytes} [Tue Aug 2 10:35:14 2022] GET /identity/v3/role_assignments?user.id=35c42cdc208f40c2bfbd0f414e36bd95&scope.system=all&role.id=33c5141e078844bbb60127f2b26540d7 => generated 227 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:14.747980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64449a10-0c95-49ed-af06-dd33ff295478 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:14.749711 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69458970-b2e0-484a-9ccb-1a71b13abaf9 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:14.750001 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69458970-b2e0-484a-9ccb-1a71b13abaf9 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:14.750280 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69458970-b2e0-484a-9ccb-1a71b13abaf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:14.778112 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-69458970-b2e0-484a-9ccb-1a71b13abaf9 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:14.779570 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-69458970-b2e0-484a-9ccb-1a71b13abaf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:14.847184 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:14.879784 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 137/285] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:14 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:14.914790 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69458970-b2e0-484a-9ccb-1a71b13abaf9 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:14.921712 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64449a10-0c95-49ed-af06-dd33ff295478 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:14.939476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:14.967750 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3d82d-58b1-4a07-aa7a-7e766cbe17b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:14.969780 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3d82d-58b1-4a07-aa7a-7e766cbe17b8 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:14.970306 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3d82d-58b1-4a07-aa7a-7e766cbe17b8 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:14.970786 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc3d82d-58b1-4a07-aa7a-7e766cbe17b8 None admin] PATH_INFO: `/v3/users/5e30988273d44931859a959d14bcffad` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:14.972684 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3d82d-58b1-4a07-aa7a-7e766cbe17b8 None admin] RBAC: Authorizing `identity:get_user(user_id=5e30988273d44931859a959d14bcffad)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:14.974858 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc3d82d-58b1-4a07-aa7a-7e766cbe17b8 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:14.977991 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 149/286] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:35:14 2022] GET /identity/v3/users/5e30988273d44931859a959d14bcffad => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:14.983089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3d82d-58b1-4a07-aa7a-7e766cbe17b8 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:15.038351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69458970-b2e0-484a-9ccb-1a71b13abaf9 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:15.076087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a36c9-2f87-4f3e-8222-570fe2466201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:15.077101 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a36c9-2f87-4f3e-8222-570fe2466201 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:15.077583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a36c9-2f87-4f3e-8222-570fe2466201 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:15.078093 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a36c9-2f87-4f3e-8222-570fe2466201 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:15.097127 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc3d82d-58b1-4a07-aa7a-7e766cbe17b8 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:15.101487 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a36c9-2f87-4f3e-8222-570fe2466201 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:15.106877 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-ec0a36c9-2f87-4f3e-8222-570fe2466201 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:35:15.106877 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:15.106877 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:15.106877 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:15.106877 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:15.106877 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:15.106877 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:15.106877 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:15.106877 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:15.106877 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:15.106877 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:15.106877 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:15.106877 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:15.106877 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:15.106877 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:15.106877 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:15.107845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:15.107845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:15.107845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:15.107845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:15.107845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:15.107845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:15.107845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:15.107845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:15.107845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:15.107845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:15.107845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:15.107845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:15.107845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:15.107845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:15.107845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:15.107845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:15.108648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:15.108648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:15.108648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:15.108648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:15.108648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:15.108648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:15.108648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:15.108648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:15.108648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:15.108648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:15.108648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:35:15.108648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:15.111827 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 138/287] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:35:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:15.117821 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a36c9-2f87-4f3e-8222-570fe2466201 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:15.145203 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1950a4-295c-4ca4-9f7f-c89c47be3c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:15.145862 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1950a4-295c-4ca4-9f7f-c89c47be3c6e None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:15.146141 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1950a4-295c-4ca4-9f7f-c89c47be3c6e None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:15.146412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1950a4-295c-4ca4-9f7f-c89c47be3c6e None admin] PATH_INFO: `/v3/projects/21b65aad6f794b3588a867ef02e946ac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:15.147620 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1950a4-295c-4ca4-9f7f-c89c47be3c6e None admin] RBAC: Authorizing `identity:get_project(project_id=21b65aad6f794b3588a867ef02e946ac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:15.149069 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f1950a4-295c-4ca4-9f7f-c89c47be3c6e None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:15.150757 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 150/288] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:35:14 2022] GET /identity/v3/projects/21b65aad6f794b3588a867ef02e946ac => generated 303 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:35:15.151303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:15.162254 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1950a4-295c-4ca4-9f7f-c89c47be3c6e None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:15.294528 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:15.313210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1950a4-295c-4ca4-9f7f-c89c47be3c6e None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:15.347073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a36c9-2f87-4f3e-8222-570fe2466201 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:15.350690 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feda8ae-4315-4237-9d48-38153c170a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:15.351626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feda8ae-4315-4237-9d48-38153c170a01 None admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:15.351908 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feda8ae-4315-4237-9d48-38153c170a01 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:15.352184 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feda8ae-4315-4237-9d48-38153c170a01 None admin] PATH_INFO: `/v3/projects/21b65aad6f794b3588a867ef02e946ac/users/5e30988273d44931859a959d14bcffad/roles/1cf433b432764fbab711f54dcf7aa81e` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:15.354193 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feda8ae-4315-4237-9d48-38153c170a01 None admin] RBAC: Authorizing `identity:create_grant(project_id=21b65aad6f794b3588a867ef02e946ac, user_id=5e30988273d44931859a959d14bcffad, role_id=1cf433b432764fbab711f54dcf7aa81e)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:15.355810 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1feda8ae-4315-4237-9d48-38153c170a01 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:15.395140 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abdeb26-0143-4a9e-a337-f44700028926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:15.395877 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdeb26-0143-4a9e-a337-f44700028926 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:15.396163 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdeb26-0143-4a9e-a337-f44700028926 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:15.396437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abdeb26-0143-4a9e-a337-f44700028926 None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:15.397225 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abdeb26-0143-4a9e-a337-f44700028926 None admin] RBAC: Adding query filter params (name=member) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:15.397532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abdeb26-0143-4a9e-a337-f44700028926 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:15.399210 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abdeb26-0143-4a9e-a337-f44700028926 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:15.408979 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 139/289] 10.222.0.115 () {56 vars in 1106 bytes} [Tue Aug 2 10:35:15 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:15.414266 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abdeb26-0143-4a9e-a337-f44700028926 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:15.430842 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 151/290] 10.222.0.115 () {58 vars in 1444 bytes} [Tue Aug 2 10:35:15 2022] PUT /identity/v3/projects/21b65aad6f794b3588a867ef02e946ac/users/5e30988273d44931859a959d14bcffad/roles/1cf433b432764fbab711f54dcf7aa81e => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:35:15.566754 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abdeb26-0143-4a9e-a337-f44700028926 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:15.618755 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da19fc95-5ddf-4746-96d3-52a40b141866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:15.619604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19fc95-5ddf-4746-96d3-52a40b141866 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:15.619971 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19fc95-5ddf-4746-96d3-52a40b141866 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:15.620250 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da19fc95-5ddf-4746-96d3-52a40b141866 None admin] PATH_INFO: `/v3/users/06535fb063c04013957d150e411bec79` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:15.622412 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da19fc95-5ddf-4746-96d3-52a40b141866 None admin] RBAC: Authorizing `identity:get_user(user_id=06535fb063c04013957d150e411bec79)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:15.623895 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da19fc95-5ddf-4746-96d3-52a40b141866 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:15.626217 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 140/291] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:35:15 2022] GET /identity/v3/users/06535fb063c04013957d150e411bec79 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:15.631727 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feda8ae-4315-4237-9d48-38153c170a01 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:15.699972 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:15.756302 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feda8ae-4315-4237-9d48-38153c170a01 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:15.793962 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147f90a-e065-4f49-bc30-bb639acb7a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:15.794589 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147f90a-e065-4f49-bc30-bb639acb7a4a None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:15.794866 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147f90a-e065-4f49-bc30-bb639acb7a4a None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:15.795163 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f147f90a-e065-4f49-bc30-bb639acb7a4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:15.868154 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f147f90a-e065-4f49-bc30-bb639acb7a4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:15.869534 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f147f90a-e065-4f49-bc30-bb639acb7a4a None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:15.931610 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:15.979253 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 152/292] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:35:15 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:35:15.982001 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da19fc95-5ddf-4746-96d3-52a40b141866 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:15.994431 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:16.120862 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da19fc95-5ddf-4746-96d3-52a40b141866 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:16.179605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba25034-50a0-437b-9847-e09dd2d8ac9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:16.180253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba25034-50a0-437b-9847-e09dd2d8ac9e None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:16.180530 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba25034-50a0-437b-9847-e09dd2d8ac9e None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:16.180799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba25034-50a0-437b-9847-e09dd2d8ac9e None admin] PATH_INFO: `/v3/projects/21b65aad6f794b3588a867ef02e946ac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:16.187868 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba25034-50a0-437b-9847-e09dd2d8ac9e None admin] RBAC: Authorizing `identity:get_project(project_id=21b65aad6f794b3588a867ef02e946ac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:16.190446 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba25034-50a0-437b-9847-e09dd2d8ac9e None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:16.192600 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 141/293] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:35:15 2022] GET /identity/v3/projects/21b65aad6f794b3588a867ef02e946ac => generated 303 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:16.198632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147f90a-e065-4f49-bc30-bb639acb7a4a None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:16.349570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f147f90a-e065-4f49-bc30-bb639acb7a4a None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:16.395477 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90fb3ba-c984-4a9e-b7c4-786da30ba8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:16.396441 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90fb3ba-c984-4a9e-b7c4-786da30ba8f8 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:16.396925 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90fb3ba-c984-4a9e-b7c4-786da30ba8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:16.397469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90fb3ba-c984-4a9e-b7c4-786da30ba8f8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:16.398536 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90fb3ba-c984-4a9e-b7c4-786da30ba8f8 None admin] RBAC: Adding query filter params (role.id=33c5141e078844bbb60127f2b26540d7, scope.project.id=21b65aad6f794b3588a867ef02e946ac, user.id=06535fb063c04013957d150e411bec79) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:16.399069 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90fb3ba-c984-4a9e-b7c4-786da30ba8f8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:16.400788 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90fb3ba-c984-4a9e-b7c4-786da30ba8f8 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:16.407568 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 153/294] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:35:16 2022] GET /identity/v3/role_assignments?user.id=06535fb063c04013957d150e411bec79&scope.project.id=21b65aad6f794b3588a867ef02e946ac&role.id=33c5141e078844bbb60127f2b26540d7 => generated 610 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:35:17.042746 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f2e79-1216-4926-871e-033109de7dd0 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:17.043411 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f2e79-1216-4926-871e-033109de7dd0 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:17.043411 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f2e79-1216-4926-871e-033109de7dd0 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:17.044951 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 142/295] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:17 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:17.051659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa510b94-9834-490d-9057-446d89e1b0ae None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:17.052172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa510b94-9834-490d-9057-446d89e1b0ae None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:17.052265 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa510b94-9834-490d-9057-446d89e1b0ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:17.071490 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-aa510b94-9834-490d-9057-446d89e1b0ae None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:17.072680 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-aa510b94-9834-490d-9057-446d89e1b0ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:17.086262 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:17.155166 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 154/296] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:17 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:17.171631 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f2e79-1216-4926-871e-033109de7dd0 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:17.261190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c570e3a-d861-4266-8a95-c63420571a6f None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:17.261474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c570e3a-d861-4266-8a95-c63420571a6f None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:17.261742 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c570e3a-d861-4266-8a95-c63420571a6f None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:17.263383 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 155/297] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:17 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:17.275646 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2b244-547a-44c6-94f3-c848725f1957 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:17.276120 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2b244-547a-44c6-94f3-c848725f1957 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:17.276492 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae2b244-547a-44c6-94f3-c848725f1957 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:17.280530 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:17.298591 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-6ae2b244-547a-44c6-94f3-c848725f1957 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:17.299905 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-6ae2b244-547a-44c6-94f3-c848725f1957 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:17.314471 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f2e79-1216-4926-871e-033109de7dd0 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:17.381628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d20cd-0f7c-4027-94fc-1bfbef86b16f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:17.383944 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d20cd-0f7c-4027-94fc-1bfbef86b16f None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:17.384843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d20cd-0f7c-4027-94fc-1bfbef86b16f None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:17.385701 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5d20cd-0f7c-4027-94fc-1bfbef86b16f None admin] PATH_INFO: `/v3/roles/member` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:17.392114 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 156/298] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:17 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:35:17.409818 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5d20cd-0f7c-4027-94fc-1bfbef86b16f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:17.420960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-bb5d20cd-0f7c-4027-94fc-1bfbef86b16f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:35:17.420960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:17.420960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:17.420960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:17.420960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:17.420960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:17.420960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:17.420960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:17.420960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:17.420960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:17.420960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:17.420960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:17.420960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:17.420960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:17.420960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:17.420960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:17.423074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:17.423074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:17.423074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:17.423074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:17.423074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:17.423074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:17.423074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:17.423074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:17.423074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:17.423074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:17.423074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:17.423074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:17.423074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:17.423074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:17.423074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:17.423074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:17.425349 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:17.425349 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:17.425349 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:17.425349 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:17.425349 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:17.425349 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:17.425349 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:17.425349 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:17.425349 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:17.425349 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:17.425349 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:35:17.425349 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:17.428503 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2b244-547a-44c6-94f3-c848725f1957 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:17.434651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 143/299] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:35:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:17.441207 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d20cd-0f7c-4027-94fc-1bfbef86b16f None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:17.567946 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:17.587167 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5d20cd-0f7c-4027-94fc-1bfbef86b16f None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:17.596397 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae2b244-547a-44c6-94f3-c848725f1957 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:17.638452 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abac6e3-380a-490a-b348-e5f0e8d4373f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:17.639085 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abac6e3-380a-490a-b348-e5f0e8d4373f None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:17.639353 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abac6e3-380a-490a-b348-e5f0e8d4373f None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:17.639623 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abac6e3-380a-490a-b348-e5f0e8d4373f None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:17.640342 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abac6e3-380a-490a-b348-e5f0e8d4373f None admin] RBAC: Adding query filter params (name=member) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:17.640460 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bbf44f-569b-4291-8f9d-b8219b609772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:17.640670 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abac6e3-380a-490a-b348-e5f0e8d4373f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:17.641067 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bbf44f-569b-4291-8f9d-b8219b609772 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:17.641355 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bbf44f-569b-4291-8f9d-b8219b609772 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:17.642198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bbf44f-569b-4291-8f9d-b8219b609772 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:17.642812 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abac6e3-380a-490a-b348-e5f0e8d4373f None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:17.643442 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bbf44f-569b-4291-8f9d-b8219b609772 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:17.644890 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bbf44f-569b-4291-8f9d-b8219b609772 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:17.645925 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:17.646538 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 157/300] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:17 2022] GET /identity/v3/domains/default => generated 200 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:17.651768 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee9a1b-1cb1-4b44-8d69-116e4d7c87b0 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:17.651768 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee9a1b-1cb1-4b44-8d69-116e4d7c87b0 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:17.651768 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee9a1b-1cb1-4b44-8d69-116e4d7c87b0 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:17.651768 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 158/301] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:17 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:17.654544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee9a1b-1cb1-4b44-8d69-116e4d7c87b0 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:17.659267 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 144/302] 10.222.0.115 () {56 vars in 1106 bytes} [Tue Aug 2 10:35:17 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:17.659267 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a0035-1c09-4a3f-ab2d-31f866e280ce None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:17.659267 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a0035-1c09-4a3f-ab2d-31f866e280ce None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:17.659267 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a0035-1c09-4a3f-ab2d-31f866e280ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:17.683357 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-937a0035-1c09-4a3f-ab2d-31f866e280ce None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:17.684612 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-937a0035-1c09-4a3f-ab2d-31f866e280ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:17.787536 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 145/303] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:17 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:35:17.788787 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a0035-1c09-4a3f-ab2d-31f866e280ce None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:17.800653 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ee9a1b-1cb1-4b44-8d69-116e4d7c87b0 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:17.847440 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c6204-b102-4c04-ac64-4acbb18c5cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:17.847980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c6204-b102-4c04-ac64-4acbb18c5cda None admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:17.848366 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c6204-b102-4c04-ac64-4acbb18c5cda None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:17.848649 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25c6204-b102-4c04-ac64-4acbb18c5cda None admin] PATH_INFO: `/v3/groups` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:17.849330 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c6204-b102-4c04-ac64-4acbb18c5cda None admin] RBAC: Authorizing `identity:create_group()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:17.850921 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25c6204-b102-4c04-ac64-4acbb18c5cda None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:17.915141 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a0035-1c09-4a3f-ab2d-31f866e280ce None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:17.943826 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 159/304] 10.222.0.115 () {60 vars in 1139 bytes} [Tue Aug 2 10:35:17 2022] POST /identity/v3/groups => generated 230 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:35:17.944682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c6204-b102-4c04-ac64-4acbb18c5cda None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:17.951138 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:17.960166 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:17.974461 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9254ec-9ce7-4521-9b50-56513a7f1e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:17.975717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9254ec-9ce7-4521-9b50-56513a7f1e9a None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:17.976223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9254ec-9ce7-4521-9b50-56513a7f1e9a None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:17.977411 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9254ec-9ce7-4521-9b50-56513a7f1e9a None admin] PATH_INFO: `/v3/users/35c42cdc208f40c2bfbd0f414e36bd95` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:17.979517 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9254ec-9ce7-4521-9b50-56513a7f1e9a None admin] RBAC: Authorizing `identity:get_user(user_id=35c42cdc208f40c2bfbd0f414e36bd95)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:17.983162 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9254ec-9ce7-4521-9b50-56513a7f1e9a None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:17.986337 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 146/305] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:35:17 2022] GET /identity/v3/users/35c42cdc208f40c2bfbd0f414e36bd95 => generated 294 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:17.992157 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9254ec-9ce7-4521-9b50-56513a7f1e9a None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:18.105378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25c6204-b102-4c04-ac64-4acbb18c5cda None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:18.150001 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4112165-d106-4e96-87d8-789563e84a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:18.150667 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4112165-d106-4e96-87d8-789563e84a61 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:18.150954 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4112165-d106-4e96-87d8-789563e84a61 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:18.151314 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4112165-d106-4e96-87d8-789563e84a61 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:18.153158 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9254ec-9ce7-4521-9b50-56513a7f1e9a None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:18.164711 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4112165-d106-4e96-87d8-789563e84a61 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:18.170480 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-e4112165-d106-4e96-87d8-789563e84a61 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:35:18.170480 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:18.170480 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:18.170480 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:18.170480 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:18.170480 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:18.170480 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:18.170480 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:18.170480 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:18.170480 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:18.170480 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:18.170480 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:18.170480 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:18.170480 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:18.170480 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:18.170480 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:18.171679 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:18.171679 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:18.171679 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:18.171679 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:18.171679 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:18.171679 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:18.171679 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:18.171679 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:18.171679 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:18.171679 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:18.171679 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:18.171679 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:18.171679 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:18.171679 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:18.171679 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:18.171679 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:18.172469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:18.172469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:18.172469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:18.172469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:18.172469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:18.172469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:18.172469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:18.172469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:18.172469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:18.172469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:18.172469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:35:18.172469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:18.172469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 160/306] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:35:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:18.174031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b7c28-0a94-4dd9-b4e9-6473885aec55 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:18.174031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b7c28-0a94-4dd9-b4e9-6473885aec55 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:18.174031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b7c28-0a94-4dd9-b4e9-6473885aec55 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:18.176170 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 161/307] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:18 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:18.181500 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3444d7-9c58-4f2a-8649-9d2c96836261 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:18.181789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3444d7-9c58-4f2a-8649-9d2c96836261 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:18.182088 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3444d7-9c58-4f2a-8649-9d2c96836261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:18.204190 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdcc26e-1540-4ba9-978d-cb9cccba14f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:18.205276 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdcc26e-1540-4ba9-978d-cb9cccba14f4 None admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:18.206296 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-3c3444d7-9c58-4f2a-8649-9d2c96836261 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:18.206397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdcc26e-1540-4ba9-978d-cb9cccba14f4 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:18.207121 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdcc26e-1540-4ba9-978d-cb9cccba14f4 None admin] PATH_INFO: `/v3/system/users/35c42cdc208f40c2bfbd0f414e36bd95/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:18.207826 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-3c3444d7-9c58-4f2a-8649-9d2c96836261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:18.209700 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdcc26e-1540-4ba9-978d-cb9cccba14f4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=35c42cdc208f40c2bfbd0f414e36bd95, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:18.211382 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdcc26e-1540-4ba9-978d-cb9cccba14f4 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:18.286357 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:18.295755 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 162/308] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:18 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:18.296600 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3444d7-9c58-4f2a-8649-9d2c96836261 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:18.316602 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 147/309] 10.222.0.115 () {58 vars in 1339 bytes} [Tue Aug 2 10:35:17 2022] PUT /identity/v3/system/users/35c42cdc208f40c2bfbd0f414e36bd95/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:35:18.333658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdcc26e-1540-4ba9-978d-cb9cccba14f4 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:18.444176 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:18.458904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3444d7-9c58-4f2a-8649-9d2c96836261 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:18.499293 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdcc26e-1540-4ba9-978d-cb9cccba14f4 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:18.513358 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb657e6-c851-4889-9410-ce4f6490b62f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:18.514008 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb657e6-c851-4889-9410-ce4f6490b62f None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:18.521714 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb657e6-c851-4889-9410-ce4f6490b62f None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:18.521714 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb657e6-c851-4889-9410-ce4f6490b62f None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:18.521714 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb657e6-c851-4889-9410-ce4f6490b62f None admin] RBAC: Adding query filter params (name=reader) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:18.521714 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb657e6-c851-4889-9410-ce4f6490b62f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:18.521714 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb657e6-c851-4889-9410-ce4f6490b62f None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:18.531173 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 163/310] 10.222.0.115 () {56 vars in 1106 bytes} [Tue Aug 2 10:35:18 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:18.535750 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb657e6-c851-4889-9410-ce4f6490b62f None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:18.561683 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabacfd-dcdd-4ee2-98b2-40a9ec5cd91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:18.562665 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabacfd-dcdd-4ee2-98b2-40a9ec5cd91a None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:18.563142 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabacfd-dcdd-4ee2-98b2-40a9ec5cd91a None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:18.564154 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deabacfd-dcdd-4ee2-98b2-40a9ec5cd91a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:18.580422 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deabacfd-dcdd-4ee2-98b2-40a9ec5cd91a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:18.586226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-deabacfd-dcdd-4ee2-98b2-40a9ec5cd91a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:35:18.586226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:18.586226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:18.586226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:18.586226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:18.586226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:18.586226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:18.586226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:18.586226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:18.586226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:18.586226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:18.586226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:18.586226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:18.586226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:18.586226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:18.586226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:18.587664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:18.587664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:18.587664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:18.587664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:18.587664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:18.587664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:18.587664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:18.587664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:18.587664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:18.587664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:18.587664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:18.587664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:18.587664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:18.587664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:18.587664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:18.587664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:18.588585 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:18.588585 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:18.588585 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:18.588585 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:18.588585 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:18.588585 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:18.588585 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:18.588585 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:18.588585 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:18.588585 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:18.588585 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:35:18.588585 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:18.588585 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 148/311] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:35:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:35:18.604505 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabacfd-dcdd-4ee2-98b2-40a9ec5cd91a None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:18.687446 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:18.740935 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb657e6-c851-4889-9410-ce4f6490b62f None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:18.786272 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d28241-90d3-4bfc-8176-73cdd420352f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:18.787191 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d28241-90d3-4bfc-8176-73cdd420352f None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:18.787812 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d28241-90d3-4bfc-8176-73cdd420352f None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:18.788232 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d28241-90d3-4bfc-8176-73cdd420352f None admin] PATH_INFO: `/v3/users/7cfc12bb707d4d1e8d1b9de873bea65b` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:18.808885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d28241-90d3-4bfc-8176-73cdd420352f None admin] RBAC: Authorizing `identity:get_user(user_id=7cfc12bb707d4d1e8d1b9de873bea65b)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:18.810640 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d28241-90d3-4bfc-8176-73cdd420352f None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:18.812461 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deabacfd-dcdd-4ee2-98b2-40a9ec5cd91a None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:18.812909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 164/312] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:35:18 2022] GET /identity/v3/users/7cfc12bb707d4d1e8d1b9de873bea65b => generated 294 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:18.820204 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d28241-90d3-4bfc-8176-73cdd420352f None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:18.854219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:18.888796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46fc724-e49d-4223-b4c0-5f3716a5c6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:18.891245 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fc724-e49d-4223-b4c0-5f3716a5c6a9 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:18.892479 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fc724-e49d-4223-b4c0-5f3716a5c6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:18.893205 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46fc724-e49d-4223-b4c0-5f3716a5c6a9 None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:18.895766 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46fc724-e49d-4223-b4c0-5f3716a5c6a9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:18.896832 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46fc724-e49d-4223-b4c0-5f3716a5c6a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:18.899684 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46fc724-e49d-4223-b4c0-5f3716a5c6a9 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:18.919137 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 149/313] 10.222.0.115 () {56 vars in 1106 bytes} [Tue Aug 2 10:35:18 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:18.921094 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46fc724-e49d-4223-b4c0-5f3716a5c6a9 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:18.982051 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d28241-90d3-4bfc-8176-73cdd420352f None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:19.001679 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:19.026432 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf96f1-65a0-4a2e-bcc6-d58eff347c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:19.027481 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf96f1-65a0-4a2e-bcc6-d58eff347c1f None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:19.027983 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf96f1-65a0-4a2e-bcc6-d58eff347c1f None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:19.028460 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bf96f1-65a0-4a2e-bcc6-d58eff347c1f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:19.029636 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf96f1-65a0-4a2e-bcc6-d58eff347c1f None admin] RBAC: Adding query filter params (role.id=1cf433b432764fbab711f54dcf7aa81e, user.id=7cfc12bb707d4d1e8d1b9de873bea65b, scope.system=all) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:19.030211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf96f1-65a0-4a2e-bcc6-d58eff347c1f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:19.032720 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bf96f1-65a0-4a2e-bcc6-d58eff347c1f None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:19.039992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 165/314] 10.222.0.115 () {56 vars in 1313 bytes} [Tue Aug 2 10:35:18 2022] GET /identity/v3/role_assignments?user.id=7cfc12bb707d4d1e8d1b9de873bea65b&scope.system=all&role.id=1cf433b432764fbab711f54dcf7aa81e => generated 227 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:19.040194 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46fc724-e49d-4223-b4c0-5f3716a5c6a9 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:19.092587 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b1c32-5be5-4e7c-bb4b-143b7461ea04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:19.093250 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b1c32-5be5-4e7c-bb4b-143b7461ea04 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:19.093526 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b1c32-5be5-4e7c-bb4b-143b7461ea04 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:19.093798 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b1c32-5be5-4e7c-bb4b-143b7461ea04 None admin] PATH_INFO: `/v3/users/5e30988273d44931859a959d14bcffad` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:19.096428 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b1c32-5be5-4e7c-bb4b-143b7461ea04 None admin] RBAC: Authorizing `identity:get_user(user_id=5e30988273d44931859a959d14bcffad)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:19.098117 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b1c32-5be5-4e7c-bb4b-143b7461ea04 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:19.100793 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 150/315] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:35:18 2022] GET /identity/v3/users/5e30988273d44931859a959d14bcffad => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:35:19.107244 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf96f1-65a0-4a2e-bcc6-d58eff347c1f None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:19.236288 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bf96f1-65a0-4a2e-bcc6-d58eff347c1f None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:19.278220 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2aaf8-a9a7-48ab-a2f8-3377b5991fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:19.279277 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2aaf8-a9a7-48ab-a2f8-3377b5991fee None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:19.279786 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2aaf8-a9a7-48ab-a2f8-3377b5991fee None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:19.280265 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2aaf8-a9a7-48ab-a2f8-3377b5991fee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:19.355500 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:19.388772 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2aaf8-a9a7-48ab-a2f8-3377b5991fee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:19.390473 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2aaf8-a9a7-48ab-a2f8-3377b5991fee None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:19.512927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 166/316] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:35:19 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:35:19.520309 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b1c32-5be5-4e7c-bb4b-143b7461ea04 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:19.576883 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:19.633112 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b1c32-5be5-4e7c-bb4b-143b7461ea04 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:19.674629 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a2a806-d899-4929-906a-f8b447db83bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:19.675316 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2a806-d899-4929-906a-f8b447db83bd None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:19.675598 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2a806-d899-4929-906a-f8b447db83bd None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:19.675869 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a2a806-d899-4929-906a-f8b447db83bd None admin] PATH_INFO: `/v3/projects/21b65aad6f794b3588a867ef02e946ac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:19.677094 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2a806-d899-4929-906a-f8b447db83bd None admin] RBAC: Authorizing `identity:get_project(project_id=21b65aad6f794b3588a867ef02e946ac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:19.678635 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a2a806-d899-4929-906a-f8b447db83bd None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:19.680808 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 151/317] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:35:19 2022] GET /identity/v3/projects/21b65aad6f794b3588a867ef02e946ac => generated 303 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:19.686799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2aaf8-a9a7-48ab-a2f8-3377b5991fee None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:19.773742 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:19.796418 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2aaf8-a9a7-48ab-a2f8-3377b5991fee None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:19.836944 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f28660f-dd9a-4e98-8d25-8fb57511fbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:19.837494 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28660f-dd9a-4e98-8d25-8fb57511fbca None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:19.837782 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28660f-dd9a-4e98-8d25-8fb57511fbca None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:19.838058 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f28660f-dd9a-4e98-8d25-8fb57511fbca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:19.838857 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f28660f-dd9a-4e98-8d25-8fb57511fbca None admin] RBAC: Adding query filter params (role.id=1cf433b432764fbab711f54dcf7aa81e, scope.project.id=21b65aad6f794b3588a867ef02e946ac, user.id=5e30988273d44931859a959d14bcffad) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:19.839199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f28660f-dd9a-4e98-8d25-8fb57511fbca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:19.840642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f28660f-dd9a-4e98-8d25-8fb57511fbca None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:19.847822 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 167/318] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:35:19 2022] GET /identity/v3/role_assignments?user.id=5e30988273d44931859a959d14bcffad&scope.project.id=21b65aad6f794b3588a867ef02e946ac&role.id=1cf433b432764fbab711f54dcf7aa81e => generated 610 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:20.433995 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9065e-fe94-4237-a1a4-b5a62b3df654 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:20.434399 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9065e-fe94-4237-a1a4-b5a62b3df654 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:20.434602 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e9065e-fe94-4237-a1a4-b5a62b3df654 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:20.436142 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 152/319] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:20 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:20.443519 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1442588-d573-4206-b3a5-55d9b02fd3de None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:20.444287 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1442588-d573-4206-b3a5-55d9b02fd3de None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:20.444287 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1442588-d573-4206-b3a5-55d9b02fd3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:20.466179 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-c1442588-d573-4206-b3a5-55d9b02fd3de None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:20.467552 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-c1442588-d573-4206-b3a5-55d9b02fd3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:20.552907 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 168/320] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:20 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:35:20.589274 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9065e-fe94-4237-a1a4-b5a62b3df654 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:20.698877 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e9065e-fe94-4237-a1a4-b5a62b3df654 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:20.717616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:20.724724 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb16c52-5b9d-4ef8-a90e-8390ee96d97b None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:20.725080 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb16c52-5b9d-4ef8-a90e-8390ee96d97b None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:20.725560 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb16c52-5b9d-4ef8-a90e-8390ee96d97b None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:20.726903 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 169/321] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:20 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:20.732487 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d6cc1-260a-41cc-9030-8a7ab7ffacbb None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:20.732773 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d6cc1-260a-41cc-9030-8a7ab7ffacbb None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:20.733058 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d6cc1-260a-41cc-9030-8a7ab7ffacbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:20.744785 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf07b8-4083-4c60-bdec-c633780e36c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:20.745425 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf07b8-4083-4c60-bdec-c633780e36c6 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:20.745698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf07b8-4083-4c60-bdec-c633780e36c6 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:20.745965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf07b8-4083-4c60-bdec-c633780e36c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:20.747122 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf07b8-4083-4c60-bdec-c633780e36c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:20.748581 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf07b8-4083-4c60-bdec-c633780e36c6 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:20.750387 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 153/322] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:20 2022] GET /identity/v3/domains/default => generated 200 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:20.753952 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-a27d6cc1-260a-41cc-9030-8a7ab7ffacbb None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:20.755250 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-a27d6cc1-260a-41cc-9030-8a7ab7ffacbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:20.756400 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf07b8-4083-4c60-bdec-c633780e36c6 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:20.847211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 170/323] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:20 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:35:20.873077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d6cc1-260a-41cc-9030-8a7ab7ffacbb None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:20.875389 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:20.880631 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf07b8-4083-4c60-bdec-c633780e36c6 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:20.930674 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a31ef-7bf2-40aa-8e2c-34bbd3697005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:20.931324 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a31ef-7bf2-40aa-8e2c-34bbd3697005 None admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:20.931601 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a31ef-7bf2-40aa-8e2c-34bbd3697005 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:20.931876 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5a31ef-7bf2-40aa-8e2c-34bbd3697005 None admin] PATH_INFO: `/v3/groups` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:20.932553 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5a31ef-7bf2-40aa-8e2c-34bbd3697005 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:20.933990 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5a31ef-7bf2-40aa-8e2c-34bbd3697005 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:20.994334 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27d6cc1-260a-41cc-9030-8a7ab7ffacbb None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:21.001529 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 154/324] 10.222.0.115 () {60 vars in 1139 bytes} [Tue Aug 2 10:35:20 2022] POST /identity/v3/groups => generated 227 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:35:21.049027 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b25d9c-4826-46d6-a467-474797eb3520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:21.050107 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b25d9c-4826-46d6-a467-474797eb3520 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:21.050593 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b25d9c-4826-46d6-a467-474797eb3520 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:21.051084 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b25d9c-4826-46d6-a467-474797eb3520 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:21.072064 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b25d9c-4826-46d6-a467-474797eb3520 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:21.082369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-31b25d9c-4826-46d6-a467-474797eb3520 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:35:21.082369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:21.082369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:21.082369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:21.082369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:21.082369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:21.082369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:21.082369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:21.082369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:21.082369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:21.082369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:21.082369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:21.082369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:21.082369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:21.082369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:21.082369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:21.083909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:21.083909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:21.083909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:21.083909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:21.083909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:21.083909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:21.083909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:21.083909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:21.083909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:21.083909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:21.083909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:21.083909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:21.083909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:21.083909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:21.083909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:21.083909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:21.084742 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:21.084742 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:21.084742 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:21.084742 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:21.084742 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:21.084742 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:21.084742 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:21.084742 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:21.084742 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:21.084742 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:21.084742 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:35:21.084742 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:21.086267 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 171/325] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:35:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:35:21.090914 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a31ef-7bf2-40aa-8e2c-34bbd3697005 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:21.166601 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:21.233763 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5a31ef-7bf2-40aa-8e2c-34bbd3697005 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:21.272411 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76feb7-7321-49fc-b1aa-02fd7dbd988c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:21.273501 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76feb7-7321-49fc-b1aa-02fd7dbd988c None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:21.273501 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76feb7-7321-49fc-b1aa-02fd7dbd988c None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:21.273950 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76feb7-7321-49fc-b1aa-02fd7dbd988c None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:21.274559 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76feb7-7321-49fc-b1aa-02fd7dbd988c None admin] RBAC: Adding query filter params (name=member) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:21.274559 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76feb7-7321-49fc-b1aa-02fd7dbd988c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:21.275934 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76feb7-7321-49fc-b1aa-02fd7dbd988c None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:21.287634 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 155/326] 10.222.0.115 () {56 vars in 1106 bytes} [Tue Aug 2 10:35:21 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:21.296038 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b25d9c-4826-46d6-a467-474797eb3520 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:21.307863 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b65bb-d734-47b4-be8a-85b30e740e12 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:21.308228 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b65bb-d734-47b4-be8a-85b30e740e12 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:21.308482 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b65bb-d734-47b4-be8a-85b30e740e12 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:21.310053 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 156/327] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:21 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:35:21.316958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34905e7-9d4a-4fb6-b7c3-272824158d52 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:21.318054 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34905e7-9d4a-4fb6-b7c3-272824158d52 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:21.318796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34905e7-9d4a-4fb6-b7c3-272824158d52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:21.348071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-a34905e7-9d4a-4fb6-b7c3-272824158d52 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:21.349716 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-a34905e7-9d4a-4fb6-b7c3-272824158d52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:21.381144 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:21.440675 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b25d9c-4826-46d6-a467-474797eb3520 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:21.451262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 157/328] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:21 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:35:21.475793 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34905e7-9d4a-4fb6-b7c3-272824158d52 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:21.487104 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c9dc88-f352-4607-bea7-517b2aa91852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:21.488720 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9dc88-f352-4607-bea7-517b2aa91852 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:21.489052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9dc88-f352-4607-bea7-517b2aa91852 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:21.489285 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c9dc88-f352-4607-bea7-517b2aa91852 None admin] PATH_INFO: `/v3/users/35c42cdc208f40c2bfbd0f414e36bd95` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:21.490606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c9dc88-f352-4607-bea7-517b2aa91852 None admin] RBAC: Authorizing `identity:get_user(user_id=35c42cdc208f40c2bfbd0f414e36bd95)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:21.492116 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c9dc88-f352-4607-bea7-517b2aa91852 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:21.494106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 172/329] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:35:21 2022] GET /identity/v3/users/35c42cdc208f40c2bfbd0f414e36bd95 => generated 294 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:21.502766 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c9dc88-f352-4607-bea7-517b2aa91852 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:21.529360 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:21.607431 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34905e7-9d4a-4fb6-b7c3-272824158d52 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:21.650718 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c9dc88-f352-4607-bea7-517b2aa91852 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:21.653651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d159d72-a867-453b-94a1-4c51a75e6b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:21.654654 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d159d72-a867-453b-94a1-4c51a75e6b98 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:21.655188 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d159d72-a867-453b-94a1-4c51a75e6b98 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:21.655689 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d159d72-a867-453b-94a1-4c51a75e6b98 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:21.669984 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d159d72-a867-453b-94a1-4c51a75e6b98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:21.676284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-0d159d72-a867-453b-94a1-4c51a75e6b98 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:35:21.676284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:21.676284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:21.676284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:21.676284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:21.676284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:21.676284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:21.676284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:21.676284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:21.676284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:21.676284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:21.676284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:21.676284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:21.676284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:21.676284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:21.676284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:21.677245 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:21.677245 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:21.677245 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:21.677245 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:21.677245 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:21.677245 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:21.677245 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:21.677245 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:21.677245 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:21.677245 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:21.677245 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:21.677245 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:21.677245 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:21.677245 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:21.677245 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:21.677245 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:21.678041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:21.678041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:21.678041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:21.678041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:21.678041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:21.678041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:21.678041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:21.678041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:21.678041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:21.678041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:21.678041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:35:21.678041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:21.681660 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 158/330] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:35:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:35:21.685531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:21.688975 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d159d72-a867-453b-94a1-4c51a75e6b98 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:21.694789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54c3e7-2942-4030-bb8d-0301c0e0b1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:21.699093 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54c3e7-2942-4030-bb8d-0301c0e0b1f4 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:21.699093 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54c3e7-2942-4030-bb8d-0301c0e0b1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:21.699093 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54c3e7-2942-4030-bb8d-0301c0e0b1f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:21.699093 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54c3e7-2942-4030-bb8d-0301c0e0b1f4 None admin] RBAC: Adding query filter params (role.id=33c5141e078844bbb60127f2b26540d7, user.id=35c42cdc208f40c2bfbd0f414e36bd95, scope.system=all) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:21.699093 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54c3e7-2942-4030-bb8d-0301c0e0b1f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:21.700361 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54c3e7-2942-4030-bb8d-0301c0e0b1f4 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:21.723857 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 173/331] 10.222.0.115 () {56 vars in 1313 bytes} [Tue Aug 2 10:35:21 2022] GET /identity/v3/role_assignments?user.id=35c42cdc208f40c2bfbd0f414e36bd95&scope.system=all&role.id=33c5141e078844bbb60127f2b26540d7 => generated 512 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:35:21.836963 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:21.841308 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d159d72-a867-453b-94a1-4c51a75e6b98 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:21.883760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349afe4-b67b-4634-91b8-dbea60414f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:21.884741 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349afe4-b67b-4634-91b8-dbea60414f14 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:21.885158 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349afe4-b67b-4634-91b8-dbea60414f14 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:21.885445 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0349afe4-b67b-4634-91b8-dbea60414f14 None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:21.886236 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349afe4-b67b-4634-91b8-dbea60414f14 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:21.886664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349afe4-b67b-4634-91b8-dbea60414f14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:21.888152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0349afe4-b67b-4634-91b8-dbea60414f14 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:21.902132 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 159/332] 10.222.0.115 () {56 vars in 1106 bytes} [Tue Aug 2 10:35:21 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:21.910409 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54c3e7-2942-4030-bb8d-0301c0e0b1f4 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:22.043082 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54c3e7-2942-4030-bb8d-0301c0e0b1f4 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:22.087288 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d80d81-51ee-4b56-a327-15f8edc0470e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:22.088123 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d80d81-51ee-4b56-a327-15f8edc0470e None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:22.088414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d80d81-51ee-4b56-a327-15f8edc0470e None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:22.088696 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d80d81-51ee-4b56-a327-15f8edc0470e None admin] PATH_INFO: `/v3/users/7cfc12bb707d4d1e8d1b9de873bea65b` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:22.089930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d80d81-51ee-4b56-a327-15f8edc0470e None admin] RBAC: Authorizing `identity:get_user(user_id=7cfc12bb707d4d1e8d1b9de873bea65b)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:22.091401 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d80d81-51ee-4b56-a327-15f8edc0470e None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:22.093347 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 174/333] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:35:21 2022] GET /identity/v3/users/7cfc12bb707d4d1e8d1b9de873bea65b => generated 294 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:22.099603 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349afe4-b67b-4634-91b8-dbea60414f14 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:22.219164 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0349afe4-b67b-4634-91b8-dbea60414f14 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:22.260838 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dfebd6-39a5-4561-a29d-5c99862469fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:22.261517 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfebd6-39a5-4561-a29d-5c99862469fb None admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:22.261797 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfebd6-39a5-4561-a29d-5c99862469fb None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:22.262073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dfebd6-39a5-4561-a29d-5c99862469fb None admin] PATH_INFO: `/v3/system/users/7cfc12bb707d4d1e8d1b9de873bea65b/roles/1cf433b432764fbab711f54dcf7aa81e` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:22.263640 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dfebd6-39a5-4561-a29d-5c99862469fb None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7cfc12bb707d4d1e8d1b9de873bea65b, role_id=1cf433b432764fbab711f54dcf7aa81e)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:22.265038 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dfebd6-39a5-4561-a29d-5c99862469fb None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:22.341979 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 160/334] 10.222.0.115 () {58 vars in 1339 bytes} [Tue Aug 2 10:35:22 2022] PUT /identity/v3/system/users/7cfc12bb707d4d1e8d1b9de873bea65b/roles/1cf433b432764fbab711f54dcf7aa81e => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:35:23.519698 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5d338-0585-412d-8537-889d11e92a67 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:23.520238 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5d338-0585-412d-8537-889d11e92a67 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:23.520238 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a5d338-0585-412d-8537-889d11e92a67 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:23.521913 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 175/335] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:23 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:23.528325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8c61e-db89-454c-ae4c-49b2644992b2 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:23.528797 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8c61e-db89-454c-ae4c-49b2644992b2 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:23.528896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad8c61e-db89-454c-ae4c-49b2644992b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:23.555324 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-5ad8c61e-db89-454c-ae4c-49b2644992b2 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:23.556546 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-5ad8c61e-db89-454c-ae4c-49b2644992b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:23.584056 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:23.645754 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 161/336] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:23 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:35:23.682434 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5d338-0585-412d-8537-889d11e92a67 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:23.762556 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:23.824176 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a5d338-0585-412d-8537-889d11e92a67 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:23.873391 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a942993d-335f-4195-af52-b1935519b348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:23.874057 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942993d-335f-4195-af52-b1935519b348 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:23.874341 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942993d-335f-4195-af52-b1935519b348 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:23.874618 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942993d-335f-4195-af52-b1935519b348 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:23.889350 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a942993d-335f-4195-af52-b1935519b348 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:23.899567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-a942993d-335f-4195-af52-b1935519b348 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:35:23.899567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:23.899567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:23.899567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:23.899567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:23.899567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:23.899567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:23.899567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:23.899567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:23.899567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:23.899567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:23.899567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:23.899567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:23.899567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:23.899567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:23.899567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:23.901031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:23.901031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:23.901031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:23.901031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:23.901031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:23.901031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:23.901031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:23.901031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:23.901031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:23.901031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:23.901031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:23.901031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:23.901031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:23.901031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:23.901031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:23.901031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:23.902542 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:23.902542 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:23.902542 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:23.902542 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:23.902542 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:23.902542 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:23.902542 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:23.902542 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:23.902542 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:23.902542 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:23.902542 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:35:23.902542 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:23.902542 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 176/337] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:35:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:23.913785 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8c61e-db89-454c-ae4c-49b2644992b2 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:24.098799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f900650-477e-44e8-9277-9dfb3f1efdfd None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:24.099369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f900650-477e-44e8-9277-9dfb3f1efdfd None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:24.099369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f900650-477e-44e8-9277-9dfb3f1efdfd None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:24.100783 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 177/338] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:24 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:24.107846 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68031778-4905-46a9-ad93-d54283c9ba6d None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:24.108148 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68031778-4905-46a9-ad93-d54283c9ba6d None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:24.108395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68031778-4905-46a9-ad93-d54283c9ba6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:24.109265 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad8c61e-db89-454c-ae4c-49b2644992b2 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:24.144120 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-68031778-4905-46a9-ad93-d54283c9ba6d None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:24.146057 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-68031778-4905-46a9-ad93-d54283c9ba6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:24.191646 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff67896-1260-4de7-99c0-7214ffa49f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:24.192443 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff67896-1260-4de7-99c0-7214ffa49f92 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:24.192727 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff67896-1260-4de7-99c0-7214ffa49f92 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:24.193120 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff67896-1260-4de7-99c0-7214ffa49f92 None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:24.194060 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff67896-1260-4de7-99c0-7214ffa49f92 None admin] RBAC: Adding query filter params (name=member) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:24.194371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff67896-1260-4de7-99c0-7214ffa49f92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:24.195798 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff67896-1260-4de7-99c0-7214ffa49f92 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:24.217512 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 162/339] 10.222.0.115 () {56 vars in 1106 bytes} [Tue Aug 2 10:35:23 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:24.227320 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff67896-1260-4de7-99c0-7214ffa49f92 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:24.264648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:24.302052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 178/340] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:24 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:35:24.348317 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68031778-4905-46a9-ad93-d54283c9ba6d None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:24.402742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff67896-1260-4de7-99c0-7214ffa49f92 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:24.458999 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:24.492794 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aa06ad-fee1-4301-8f08-8f7d6276a15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:24.493835 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa06ad-fee1-4301-8f08-8f7d6276a15d None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:24.494315 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa06ad-fee1-4301-8f08-8f7d6276a15d None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:24.494852 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aa06ad-fee1-4301-8f08-8f7d6276a15d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:24.570352 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68031778-4905-46a9-ad93-d54283c9ba6d None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:24.636368 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aa06ad-fee1-4301-8f08-8f7d6276a15d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:24.636934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b0fdfb-7112-4429-a39e-9f486a53aef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:24.637290 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0fdfb-7112-4429-a39e-9f486a53aef4 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:24.637574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0fdfb-7112-4429-a39e-9f486a53aef4 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:24.637853 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0fdfb-7112-4429-a39e-9f486a53aef4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:24.639629 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aa06ad-fee1-4301-8f08-8f7d6276a15d None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:24.667961 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b0fdfb-7112-4429-a39e-9f486a53aef4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:24.673919 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-d2b0fdfb-7112-4429-a39e-9f486a53aef4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:35:24.673919 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:24.673919 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:24.673919 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:24.673919 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:24.673919 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:24.673919 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:24.673919 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:24.673919 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:24.673919 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:24.673919 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:24.673919 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:24.673919 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:24.673919 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:24.673919 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:24.673919 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:24.674869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:24.674869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:24.674869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:24.674869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:24.674869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:24.674869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:24.674869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:24.674869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:24.674869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:24.674869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:24.674869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:24.674869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:24.674869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:24.674869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:24.674869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:24.674869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:24.675698 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:24.675698 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:24.675698 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:24.675698 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:24.675698 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:24.675698 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:24.675698 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:24.675698 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:24.675698 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:24.675698 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:24.675698 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:35:24.675698 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:24.675698 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 179/341] 10.222.0.115 () {56 vars in 1119 bytes} [Tue Aug 2 10:35:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:35:24.684817 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:24.695980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b0fdfb-7112-4429-a39e-9f486a53aef4 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:24.842799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 163/342] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:35:24 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:35:24.845458 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819f7ad-0c99-4943-a2bf-3430dab60d55 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:24.845749 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819f7ad-0c99-4943-a2bf-3430dab60d55 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:24.846028 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819f7ad-0c99-4943-a2bf-3430dab60d55 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:24.849616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 164/343] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:24 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:24.850586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2819f7ad-0c99-4943-a2bf-3430dab60d55 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:24.939316 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b0fdfb-7112-4429-a39e-9f486a53aef4 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:24.945575 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:24.979794 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277b7950-1038-4a2b-9bd8-045b2509dd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:24.980459 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b7950-1038-4a2b-9bd8-045b2509dd93 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:24.980734 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b7950-1038-4a2b-9bd8-045b2509dd93 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:24.981010 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277b7950-1038-4a2b-9bd8-045b2509dd93 None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:24.982813 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b7950-1038-4a2b-9bd8-045b2509dd93 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:24.983171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b7950-1038-4a2b-9bd8-045b2509dd93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:24.984636 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277b7950-1038-4a2b-9bd8-045b2509dd93 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:24.997154 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 180/344] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:35:24 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:35:24.998454 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea62ce5-2fc2-4175-9376-1bd9e866af65 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:24.998742 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea62ce5-2fc2-4175-9376-1bd9e866af65 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:24.999020 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea62ce5-2fc2-4175-9376-1bd9e866af65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:25.020892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2819f7ad-0c99-4943-a2bf-3430dab60d55 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:25.025929 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-8ea62ce5-2fc2-4175-9376-1bd9e866af65 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:25.027175 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-8ea62ce5-2fc2-4175-9376-1bd9e866af65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:25.050780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:25.088969 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c23510-b257-4baa-b702-a5bc67d28353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:25.090164 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c23510-b257-4baa-b702-a5bc67d28353 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:25.090940 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c23510-b257-4baa-b702-a5bc67d28353 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:25.091653 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c23510-b257-4baa-b702-a5bc67d28353 None admin] PATH_INFO: `/v3/projects/459ff8f521284cf8b3ec858bfb1ba630` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:25.096219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c23510-b257-4baa-b702-a5bc67d28353 None admin] RBAC: Authorizing `identity:get_project(project_id=459ff8f521284cf8b3ec858bfb1ba630)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:25.098716 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c23510-b257-4baa-b702-a5bc67d28353 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:25.101736 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 165/345] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:35:24 2022] GET /identity/v3/projects/459ff8f521284cf8b3ec858bfb1ba630 => generated 299 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:25.103400 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c23510-b257-4baa-b702-a5bc67d28353 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:25.117690 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 181/346] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:24 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:25.118555 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea62ce5-2fc2-4175-9376-1bd9e866af65 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:25.261093 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c23510-b257-4baa-b702-a5bc67d28353 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:25.264524 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:25.269890 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea62ce5-2fc2-4175-9376-1bd9e866af65 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:25.311902 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f80dd4-e2bc-4723-baa4-849765c32581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:25.312758 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f80dd4-e2bc-4723-baa4-849765c32581 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:25.313419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f80dd4-e2bc-4723-baa4-849765c32581 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:25.313822 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f80dd4-e2bc-4723-baa4-849765c32581 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:25.317414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad0698-eb56-46a4-8bc8-ac929350fe63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:25.318458 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad0698-eb56-46a4-8bc8-ac929350fe63 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:25.319012 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad0698-eb56-46a4-8bc8-ac929350fe63 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:25.319519 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ad0698-eb56-46a4-8bc8-ac929350fe63 None admin] PATH_INFO: `/v3/groups/090819f8b1a8447ca66bc3974d538dfb` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:25.327722 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ad0698-eb56-46a4-8bc8-ac929350fe63 None admin] RBAC: Authorizing `identity:get_group(group_id=090819f8b1a8447ca66bc3974d538dfb)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:25.330191 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ad0698-eb56-46a4-8bc8-ac929350fe63 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:25.333131 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 182/347] 10.222.0.115 () {56 vars in 1185 bytes} [Tue Aug 2 10:35:25 2022] GET /identity/v3/groups/090819f8b1a8447ca66bc3974d538dfb => generated 227 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:25.334332 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad0698-eb56-46a4-8bc8-ac929350fe63 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:25.397097 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:25.442023 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f80dd4-e2bc-4723-baa4-849765c32581 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:25.444581 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f80dd4-e2bc-4723-baa4-849765c32581 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:25.475529 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ad0698-eb56-46a4-8bc8-ac929350fe63 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:25.522733 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57331275-61dc-4382-8066-aba007404de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:25.523701 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57331275-61dc-4382-8066-aba007404de5 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:25.524133 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57331275-61dc-4382-8066-aba007404de5 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:25.524573 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57331275-61dc-4382-8066-aba007404de5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:25.539531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57331275-61dc-4382-8066-aba007404de5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:25.545594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-57331275-61dc-4382-8066-aba007404de5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:35:25.545594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:25.545594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:25.545594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:25.545594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:25.545594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:25.545594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:25.545594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:25.545594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:25.545594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:25.545594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:25.545594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:25.545594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:25.545594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:25.545594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:25.545594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:25.546763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:25.546763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:25.546763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:25.546763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:25.546763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:25.546763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:25.546763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:25.546763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:25.546763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:25.546763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:25.546763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:25.546763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:25.546763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:25.546763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:25.546763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:25.546763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:25.547994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:25.547994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:25.547994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:25.547994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:25.547994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:25.547994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:25.547994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:25.547994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:25.547994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:25.547994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:25.547994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:35:25.547994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:25.550412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 183/348] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:35:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:25.551833 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57331275-61dc-4382-8066-aba007404de5 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:25.592754 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 166/349] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:35:25 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:35:25.593936 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f80dd4-e2bc-4723-baa4-849765c32581 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:25.604074 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:25.707744 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57331275-61dc-4382-8066-aba007404de5 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:25.715583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:25.733260 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f80dd4-e2bc-4723-baa4-849765c32581 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:25.747281 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c99ec0-0b3b-4560-8109-2424951c51a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:25.748519 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c99ec0-0b3b-4560-8109-2424951c51a1 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:25.749133 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c99ec0-0b3b-4560-8109-2424951c51a1 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:25.749557 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c99ec0-0b3b-4560-8109-2424951c51a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:25.751119 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c99ec0-0b3b-4560-8109-2424951c51a1 None admin] RBAC: Adding query filter params (group.id=090819f8b1a8447ca66bc3974d538dfb, role.id=33c5141e078844bbb60127f2b26540d7, scope.project.id=459ff8f521284cf8b3ec858bfb1ba630) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:25.751872 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c99ec0-0b3b-4560-8109-2424951c51a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:25.753610 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c99ec0-0b3b-4560-8109-2424951c51a1 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:25.759792 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 184/350] 10.222.0.115 () {56 vars in 1381 bytes} [Tue Aug 2 10:35:25 2022] GET /identity/v3/role_assignments?group.id=090819f8b1a8447ca66bc3974d538dfb&scope.project.id=459ff8f521284cf8b3ec858bfb1ba630&role.id=33c5141e078844bbb60127f2b26540d7 => generated 261 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:25.760800 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c99ec0-0b3b-4560-8109-2424951c51a1 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:25.782594 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd854f16-751b-4239-bff1-78b161630fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:25.783793 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd854f16-751b-4239-bff1-78b161630fe9 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:25.784272 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd854f16-751b-4239-bff1-78b161630fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:25.784692 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd854f16-751b-4239-bff1-78b161630fe9 None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:25.786239 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd854f16-751b-4239-bff1-78b161630fe9 None admin] RBAC: Adding query filter params (name=member) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:25.787006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd854f16-751b-4239-bff1-78b161630fe9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:25.790457 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd854f16-751b-4239-bff1-78b161630fe9 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:25.803387 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 167/351] 10.222.0.115 () {56 vars in 1106 bytes} [Tue Aug 2 10:35:25 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:25.805246 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e9b1e-5e9a-443c-811a-5c3bd2443a4e None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:25.805749 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e9b1e-5e9a-443c-811a-5c3bd2443a4e None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:25.806165 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812e9b1e-5e9a-443c-811a-5c3bd2443a4e None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:25.812714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 168/352] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:25 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:25.816662 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e9b1e-5e9a-443c-811a-5c3bd2443a4e None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:25.903743 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:25.929107 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c99ec0-0b3b-4560-8109-2424951c51a1 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:25.992037 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812e9b1e-5e9a-443c-811a-5c3bd2443a4e None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:25.998489 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749c4353-f273-469e-a059-aff203b22535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:25.999193 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c4353-f273-469e-a059-aff203b22535 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:25.999481 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c4353-f273-469e-a059-aff203b22535 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:25.999757 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749c4353-f273-469e-a059-aff203b22535 None admin] PATH_INFO: `/v3/projects/459ff8f521284cf8b3ec858bfb1ba630` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:26.001240 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749c4353-f273-469e-a059-aff203b22535 None admin] RBAC: Authorizing `identity:get_project(project_id=459ff8f521284cf8b3ec858bfb1ba630)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:26.002783 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749c4353-f273-469e-a059-aff203b22535 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:26.004902 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 185/353] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:35:25 2022] GET /identity/v3/projects/459ff8f521284cf8b3ec858bfb1ba630 => generated 299 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:26.006110 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d19a53-0257-4274-b19d-01c92a1ee309 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:26.006396 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d19a53-0257-4274-b19d-01c92a1ee309 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:26.006674 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d19a53-0257-4274-b19d-01c92a1ee309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:26.029791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-d7d19a53-0257-4274-b19d-01c92a1ee309 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:26.031016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-d7d19a53-0257-4274-b19d-01c92a1ee309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:26.036718 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a63331b-40cb-4d94-9bf5-32afd92fa5b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:26.037290 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63331b-40cb-4d94-9bf5-32afd92fa5b5 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:26.037564 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63331b-40cb-4d94-9bf5-32afd92fa5b5 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:26.037833 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a63331b-40cb-4d94-9bf5-32afd92fa5b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:26.058348 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:26.128569 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a63331b-40cb-4d94-9bf5-32afd92fa5b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:26.130109 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a63331b-40cb-4d94-9bf5-32afd92fa5b5 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:26.140211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 186/354] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:26 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:26.141584 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d19a53-0257-4274-b19d-01c92a1ee309 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:26.256217 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 169/355] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:35:25 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:35:26.257971 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b44a20-ef98-4e01-a86f-a5b98e68f010 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:26.258586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b44a20-ef98-4e01-a86f-a5b98e68f010 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:26.263161 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b44a20-ef98-4e01-a86f-a5b98e68f010 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:26.265392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 170/356] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:26 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:26.267388 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b44a20-ef98-4e01-a86f-a5b98e68f010 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:26.289804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:26.312750 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d19a53-0257-4274-b19d-01c92a1ee309 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:26.354513 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:26.360874 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc5916-1a68-4b3e-bb90-c714ed7ae1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:26.361771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc5916-1a68-4b3e-bb90-c714ed7ae1c3 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:26.362057 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc5916-1a68-4b3e-bb90-c714ed7ae1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:26.362337 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc5916-1a68-4b3e-bb90-c714ed7ae1c3 None admin] PATH_INFO: `/v3/groups/090819f8b1a8447ca66bc3974d538dfb` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:26.363543 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc5916-1a68-4b3e-bb90-c714ed7ae1c3 None admin] RBAC: Authorizing `identity:get_group(group_id=090819f8b1a8447ca66bc3974d538dfb)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:26.365216 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc5916-1a68-4b3e-bb90-c714ed7ae1c3 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:26.371266 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 187/357] 10.222.0.115 () {56 vars in 1185 bytes} [Tue Aug 2 10:35:26 2022] GET /identity/v3/groups/090819f8b1a8447ca66bc3974d538dfb => generated 227 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:26.372134 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc5916-1a68-4b3e-bb90-c714ed7ae1c3 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:26.429782 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b44a20-ef98-4e01-a86f-a5b98e68f010 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:26.518086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc5916-1a68-4b3e-bb90-c714ed7ae1c3 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:26.521340 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccba2720-2da6-4aaf-9bf1-552f6043f1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:26.522267 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba2720-2da6-4aaf-9bf1-552f6043f1db None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:26.522686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba2720-2da6-4aaf-9bf1-552f6043f1db None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:26.523206 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba2720-2da6-4aaf-9bf1-552f6043f1db None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:26.544253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccba2720-2da6-4aaf-9bf1-552f6043f1db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:26.551404 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-ccba2720-2da6-4aaf-9bf1-552f6043f1db None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:35:26.551404 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:26.551404 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:26.551404 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:26.551404 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:26.551404 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:26.551404 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:26.551404 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:26.551404 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:26.551404 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:26.551404 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:26.551404 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:26.551404 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:26.551404 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:26.551404 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:26.551404 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:26.552351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:26.552351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:26.552351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:26.552351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:26.552351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:26.552351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:26.552351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:26.552351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:26.552351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:26.552351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:26.552351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:26.552351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:26.552351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:26.552351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:26.552351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:26.552351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:26.559958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:26.559958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:26.559958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:26.559958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:26.559958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:26.559958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:26.559958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:26.559958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:26.559958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:26.559958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:26.559958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:35:26.559958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:26.559958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 171/358] 10.222.0.115 () {56 vars in 1119 bytes} [Tue Aug 2 10:35:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:26.559958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9709c-282c-481e-848c-44d58ccae371 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:26.560676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9709c-282c-481e-848c-44d58ccae371 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:26.560676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9709c-282c-481e-848c-44d58ccae371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:26.569987 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e9a22-ada5-4301-bbf9-7aa18d13784d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:26.570795 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e9a22-ada5-4301-bbf9-7aa18d13784d None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:26.571184 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e9a22-ada5-4301-bbf9-7aa18d13784d None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:26.571470 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357e9a22-ada5-4301-bbf9-7aa18d13784d None admin] PATH_INFO: `/v3/projects/21b65aad6f794b3588a867ef02e946ac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:26.573921 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e9a22-ada5-4301-bbf9-7aa18d13784d None admin] RBAC: Authorizing `identity:get_project(project_id=21b65aad6f794b3588a867ef02e946ac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:26.580460 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357e9a22-ada5-4301-bbf9-7aa18d13784d None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:26.583735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 188/359] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:35:26 2022] GET /identity/v3/projects/21b65aad6f794b3588a867ef02e946ac => generated 303 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:26.584350 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e9a22-ada5-4301-bbf9-7aa18d13784d None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:26.591915 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-27f9709c-282c-481e-848c-44d58ccae371 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:26.593220 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-27f9709c-282c-481e-848c-44d58ccae371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:26.616750 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:26.696775 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 172/360] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:26 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:26.698069 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa848cf7-31fa-4560-9a0f-ce6bb2131362 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:26.698357 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa848cf7-31fa-4560-9a0f-ce6bb2131362 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:26.698663 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa848cf7-31fa-4560-9a0f-ce6bb2131362 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:26.700720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 173/361] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:26 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:26.701637 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa848cf7-31fa-4560-9a0f-ce6bb2131362 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:26.759185 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:26.765087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357e9a22-ada5-4301-bbf9-7aa18d13784d None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:26.816031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c4bf9-d0b6-4969-91f8-6719c22196d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:26.816886 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c4bf9-d0b6-4969-91f8-6719c22196d5 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:26.817500 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c4bf9-d0b6-4969-91f8-6719c22196d5 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:26.817944 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c4bf9-d0b6-4969-91f8-6719c22196d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:26.818892 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c4bf9-d0b6-4969-91f8-6719c22196d5 None admin] RBAC: Adding query filter params (group.id=090819f8b1a8447ca66bc3974d538dfb, role.id=a4213d713b6f49d78d7bbb9484c9312f, scope.project.id=459ff8f521284cf8b3ec858bfb1ba630) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:26.819354 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c4bf9-d0b6-4969-91f8-6719c22196d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:26.821557 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c4bf9-d0b6-4969-91f8-6719c22196d5 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:26.835305 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 189/362] 10.222.0.115 () {56 vars in 1381 bytes} [Tue Aug 2 10:35:26 2022] GET /identity/v3/role_assignments?group.id=090819f8b1a8447ca66bc3974d538dfb&scope.project.id=459ff8f521284cf8b3ec858bfb1ba630&role.id=a4213d713b6f49d78d7bbb9484c9312f => generated 261 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:26.836316 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c4bf9-d0b6-4969-91f8-6719c22196d5 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:26.869168 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa848cf7-31fa-4560-9a0f-ce6bb2131362 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:26.923069 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7495743a-61bd-4450-89af-d8db9649eec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:26.924635 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495743a-61bd-4450-89af-d8db9649eec2 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:26.925015 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495743a-61bd-4450-89af-d8db9649eec2 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:26.925439 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7495743a-61bd-4450-89af-d8db9649eec2 None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:26.926285 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495743a-61bd-4450-89af-d8db9649eec2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:26.927351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495743a-61bd-4450-89af-d8db9649eec2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:26.928405 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7495743a-61bd-4450-89af-d8db9649eec2 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:26.945300 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 174/363] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:35:26 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:26.947656 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf85e8-09bf-4a48-9d48-f468e6223a99 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:26.948096 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf85e8-09bf-4a48-9d48-f468e6223a99 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:26.948467 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bf85e8-09bf-4a48-9d48-f468e6223a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:26.975946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-b8bf85e8-09bf-4a48-9d48-f468e6223a99 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:26.977234 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-b8bf85e8-09bf-4a48-9d48-f468e6223a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:26.997382 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:27.026177 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c4bf9-d0b6-4969-91f8-6719c22196d5 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:27.087458 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 175/364] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:26 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:27.088525 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf85e8-09bf-4a48-9d48-f468e6223a99 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:27.089805 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5481f-842e-4c65-94ce-65f1bc830f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:27.090977 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5481f-842e-4c65-94ce-65f1bc830f33 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:27.091808 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5481f-842e-4c65-94ce-65f1bc830f33 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:27.092159 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5481f-842e-4c65-94ce-65f1bc830f33 None admin] PATH_INFO: `/v3/groups/090819f8b1a8447ca66bc3974d538dfb` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:27.093718 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5481f-842e-4c65-94ce-65f1bc830f33 None admin] RBAC: Authorizing `identity:get_group(group_id=090819f8b1a8447ca66bc3974d538dfb)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:27.097616 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5481f-842e-4c65-94ce-65f1bc830f33 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:27.099793 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 190/365] 10.222.0.115 () {56 vars in 1185 bytes} [Tue Aug 2 10:35:26 2022] GET /identity/v3/groups/090819f8b1a8447ca66bc3974d538dfb => generated 227 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:27.100748 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5481f-842e-4c65-94ce-65f1bc830f33 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:27.131935 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:27.259007 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5481f-842e-4c65-94ce-65f1bc830f33 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:27.262960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bf85e8-09bf-4a48-9d48-f468e6223a99 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:27.298183 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc1e75-8e0d-41b9-a508-4240cdfe73a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:27.298933 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc1e75-8e0d-41b9-a508-4240cdfe73a5 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:27.299256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc1e75-8e0d-41b9-a508-4240cdfe73a5 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:27.299534 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cc1e75-8e0d-41b9-a508-4240cdfe73a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:27.309299 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490cd45c-a18f-4e8d-9e21-2bf6fd2abbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:27.309947 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490cd45c-a18f-4e8d-9e21-2bf6fd2abbe5 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:27.310258 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490cd45c-a18f-4e8d-9e21-2bf6fd2abbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:27.310491 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490cd45c-a18f-4e8d-9e21-2bf6fd2abbe5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:27.323695 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:27.324813 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490cd45c-a18f-4e8d-9e21-2bf6fd2abbe5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:27.331638 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-490cd45c-a18f-4e8d-9e21-2bf6fd2abbe5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:35:27.331638 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:27.331638 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:27.331638 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:27.331638 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:27.331638 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:27.331638 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:27.331638 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:27.331638 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:27.331638 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:27.331638 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:27.331638 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:27.331638 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:27.331638 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:27.331638 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:27.331638 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:27.337940 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:27.337940 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:27.337940 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:27.337940 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:27.337940 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:27.337940 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:27.337940 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:27.337940 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:27.337940 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:27.337940 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:27.337940 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:27.337940 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:27.337940 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:27.337940 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:27.337940 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:27.337940 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:27.338776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:27.338776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:27.338776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:27.338776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:27.338776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:27.338776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:27.338776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:27.338776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:27.338776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:27.338776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:27.338776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 02 10:35:27.338776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:27.338776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 176/366] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:35:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:27.338776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490cd45c-a18f-4e8d-9e21-2bf6fd2abbe5 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:27.407797 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cc1e75-8e0d-41b9-a508-4240cdfe73a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:27.409536 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cc1e75-8e0d-41b9-a508-4240cdfe73a5 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:27.480451 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:27.497696 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490cd45c-a18f-4e8d-9e21-2bf6fd2abbe5 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:27.544556 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ed523-218f-4266-8313-e05f4745c230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:27.545738 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ed523-218f-4266-8313-e05f4745c230 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:27.546219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ed523-218f-4266-8313-e05f4745c230 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:27.546815 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9ed523-218f-4266-8313-e05f4745c230 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:27.549474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9ed523-218f-4266-8313-e05f4745c230 None admin] RBAC: Adding query filter params (group.id=090819f8b1a8447ca66bc3974d538dfb, role.id=33c5141e078844bbb60127f2b26540d7, scope.project.id=21b65aad6f794b3588a867ef02e946ac) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:27.550237 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9ed523-218f-4266-8313-e05f4745c230 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:27.552357 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9ed523-218f-4266-8313-e05f4745c230 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:27.560060 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 191/367] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:35:27 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:35:27.560911 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc1e75-8e0d-41b9-a508-4240cdfe73a5 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:27.561546 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 177/368] 10.222.0.115 () {56 vars in 1381 bytes} [Tue Aug 2 10:35:27 2022] GET /identity/v3/role_assignments?group.id=090819f8b1a8447ca66bc3974d538dfb&scope.project.id=21b65aad6f794b3588a867ef02e946ac&role.id=33c5141e078844bbb60127f2b26540d7 => generated 261 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:27.562583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ed523-218f-4266-8313-e05f4745c230 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:27.682735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:27.702812 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9ed523-218f-4266-8313-e05f4745c230 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:27.742583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28867ca-8a8d-4e59-b537-b6b9f4655c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:27.743592 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28867ca-8a8d-4e59-b537-b6b9f4655c60 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:27.744051 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28867ca-8a8d-4e59-b537-b6b9f4655c60 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:27.744502 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28867ca-8a8d-4e59-b537-b6b9f4655c60 None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:27.745445 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28867ca-8a8d-4e59-b537-b6b9f4655c60 None admin] RBAC: Adding query filter params (name=reader) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:27.745986 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28867ca-8a8d-4e59-b537-b6b9f4655c60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:27.747620 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28867ca-8a8d-4e59-b537-b6b9f4655c60 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:27.757492 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cc1e75-8e0d-41b9-a508-4240cdfe73a5 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:27.758886 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 178/369] 10.222.0.115 () {56 vars in 1106 bytes} [Tue Aug 2 10:35:27 2022] GET /identity/v3/roles?name=reader => generated 343 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:27.759995 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28867ca-8a8d-4e59-b537-b6b9f4655c60 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:27.798553 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:27.811490 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec690fe-c444-4622-b7a5-7c2bd98ea755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:27.812247 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec690fe-c444-4622-b7a5-7c2bd98ea755 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:27.823557 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec690fe-c444-4622-b7a5-7c2bd98ea755 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:27.823557 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec690fe-c444-4622-b7a5-7c2bd98ea755 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:27.827213 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec690fe-c444-4622-b7a5-7c2bd98ea755 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:27.834486 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-0ec690fe-c444-4622-b7a5-7c2bd98ea755 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:35:27.834486 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:27.834486 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:27.834486 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:27.834486 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:27.834486 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:27.834486 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:27.834486 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:27.834486 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:27.834486 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:27.834486 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:27.834486 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:27.834486 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:27.834486 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:27.834486 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:27.834486 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:27.836984 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:27.836984 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:27.836984 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:27.836984 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:27.836984 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:27.836984 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:27.836984 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:27.836984 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:27.836984 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:27.836984 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:27.836984 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:27.836984 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:27.836984 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:27.836984 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:27.836984 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:27.836984 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:27.838105 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:27.838105 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:27.838105 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:27.838105 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:27.838105 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:27.838105 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:27.838105 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:27.838105 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:27.838105 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:27.838105 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:27.838105 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:35:27.838105 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:27.838105 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 192/370] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:35:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:27.838105 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec690fe-c444-4622-b7a5-7c2bd98ea755 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:27.995902 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec690fe-c444-4622-b7a5-7c2bd98ea755 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:28.015338 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:28.027712 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28867ca-8a8d-4e59-b537-b6b9f4655c60 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:28.068990 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01376283-54c7-4c0b-b3ba-aab304b8bd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:28.069657 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01376283-54c7-4c0b-b3ba-aab304b8bd89 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:28.069979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01376283-54c7-4c0b-b3ba-aab304b8bd89 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:28.070272 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01376283-54c7-4c0b-b3ba-aab304b8bd89 None admin] PATH_INFO: `/v3/users/7cfc12bb707d4d1e8d1b9de873bea65b` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:28.072743 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01376283-54c7-4c0b-b3ba-aab304b8bd89 None admin] RBAC: Authorizing `identity:get_user(user_id=7cfc12bb707d4d1e8d1b9de873bea65b)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:28.074232 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01376283-54c7-4c0b-b3ba-aab304b8bd89 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:28.075944 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 193/371] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:35:27 2022] GET /identity/v3/users/7cfc12bb707d4d1e8d1b9de873bea65b => generated 294 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:28.076708 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01376283-54c7-4c0b-b3ba-aab304b8bd89 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:28.105202 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c15f3b-5a76-4316-90fc-6578023a5e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:28.105863 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c15f3b-5a76-4316-90fc-6578023a5e40 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:28.106145 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c15f3b-5a76-4316-90fc-6578023a5e40 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:28.106419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c15f3b-5a76-4316-90fc-6578023a5e40 None admin] PATH_INFO: `/v3/projects/21b65aad6f794b3588a867ef02e946ac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:28.108413 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c15f3b-5a76-4316-90fc-6578023a5e40 None admin] RBAC: Authorizing `identity:get_project(project_id=21b65aad6f794b3588a867ef02e946ac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:28.110427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c15f3b-5a76-4316-90fc-6578023a5e40 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:28.113812 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 179/372] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:35:27 2022] GET /identity/v3/projects/21b65aad6f794b3588a867ef02e946ac => generated 303 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:28.114619 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c15f3b-5a76-4316-90fc-6578023a5e40 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:28.231599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:28.254406 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c15f3b-5a76-4316-90fc-6578023a5e40 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:28.290203 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f5535-9b29-489b-a36e-c466710a63ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:28.290835 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f5535-9b29-489b-a36e-c466710a63ef None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:28.291150 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f5535-9b29-489b-a36e-c466710a63ef None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:28.291454 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360f5535-9b29-489b-a36e-c466710a63ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:28.292249 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f5535-9b29-489b-a36e-c466710a63ef None admin] RBAC: Adding query filter params (role.id=1cf433b432764fbab711f54dcf7aa81e, user.id=7cfc12bb707d4d1e8d1b9de873bea65b, scope.system=all) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:28.292569 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f5535-9b29-489b-a36e-c466710a63ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:28.294073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360f5535-9b29-489b-a36e-c466710a63ef None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:28.308247 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 180/373] 10.222.0.115 () {56 vars in 1313 bytes} [Tue Aug 2 10:35:28 2022] GET /identity/v3/role_assignments?user.id=7cfc12bb707d4d1e8d1b9de873bea65b&scope.system=all&role.id=1cf433b432764fbab711f54dcf7aa81e => generated 512 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:28.309148 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f5535-9b29-489b-a36e-c466710a63ef None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:28.317249 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01376283-54c7-4c0b-b3ba-aab304b8bd89 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:28.423842 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d6a03-270c-4241-978d-77093e4301ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:28.424487 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d6a03-270c-4241-978d-77093e4301ed None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:28.424776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d6a03-270c-4241-978d-77093e4301ed None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:28.425053 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d6a03-270c-4241-978d-77093e4301ed None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:28.425805 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d6a03-270c-4241-978d-77093e4301ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:28.426114 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d6a03-270c-4241-978d-77093e4301ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:28.426821 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360f5535-9b29-489b-a36e-c466710a63ef None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:28.427570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11d6a03-270c-4241-978d-77093e4301ed None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:28.453153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 194/374] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:35:28 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:28.454682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b781603-dae1-47e5-80f4-c0cd51fcc603 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:28.455579 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b781603-dae1-47e5-80f4-c0cd51fcc603 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:28.455579 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b781603-dae1-47e5-80f4-c0cd51fcc603 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:28.456835 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 195/375] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:28.462010 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b781603-dae1-47e5-80f4-c0cd51fcc603 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:28.469750 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20d1d33-b6d1-438e-a031-5b329590a96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:28.470554 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d1d33-b6d1-438e-a031-5b329590a96f None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:28.470838 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d1d33-b6d1-438e-a031-5b329590a96f None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:28.471163 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20d1d33-b6d1-438e-a031-5b329590a96f None admin] PATH_INFO: `/v3/groups/090819f8b1a8447ca66bc3974d538dfb` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:28.472457 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d1d33-b6d1-438e-a031-5b329590a96f None admin] RBAC: Authorizing `identity:get_group(group_id=090819f8b1a8447ca66bc3974d538dfb)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:28.473946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20d1d33-b6d1-438e-a031-5b329590a96f None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:28.475658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 181/376] 10.222.0.115 () {56 vars in 1185 bytes} [Tue Aug 2 10:35:28 2022] GET /identity/v3/groups/090819f8b1a8447ca66bc3974d538dfb => generated 227 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:28.476837 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414db4ea-0ef8-4494-9ee9-2c7ccbad9264 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:28.477121 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414db4ea-0ef8-4494-9ee9-2c7ccbad9264 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:28.477394 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414db4ea-0ef8-4494-9ee9-2c7ccbad9264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:28.497814 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-414db4ea-0ef8-4494-9ee9-2c7ccbad9264 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:28.499083 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-414db4ea-0ef8-4494-9ee9-2c7ccbad9264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:28.502290 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:28.578013 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 182/377] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:28 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:28.579271 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414db4ea-0ef8-4494-9ee9-2c7ccbad9264 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:28.599708 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:28.698447 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b781603-dae1-47e5-80f4-c0cd51fcc603 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:28.707302 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414db4ea-0ef8-4494-9ee9-2c7ccbad9264 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:28.734777 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7f7e1-62a7-4efd-8638-088ccffc5fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:28.735759 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7f7e1-62a7-4efd-8638-088ccffc5fee None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:28.736657 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7f7e1-62a7-4efd-8638-088ccffc5fee None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:28.737041 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca7f7e1-62a7-4efd-8638-088ccffc5fee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:28.747284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398ba67-e7af-4237-bb26-8a1bcbb26cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:28.748230 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398ba67-e7af-4237-bb26-8a1bcbb26cd7 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:28.748687 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398ba67-e7af-4237-bb26-8a1bcbb26cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:28.749202 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f398ba67-e7af-4237-bb26-8a1bcbb26cd7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:28.751155 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f398ba67-e7af-4237-bb26-8a1bcbb26cd7 None admin] RBAC: Adding query filter params (group.id=090819f8b1a8447ca66bc3974d538dfb, role.id=a4213d713b6f49d78d7bbb9484c9312f, scope.project.id=21b65aad6f794b3588a867ef02e946ac) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:28.752058 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f398ba67-e7af-4237-bb26-8a1bcbb26cd7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:28.753789 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f398ba67-e7af-4237-bb26-8a1bcbb26cd7 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:28.761200 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 183/378] 10.222.0.115 () {56 vars in 1381 bytes} [Tue Aug 2 10:35:28 2022] GET /identity/v3/role_assignments?group.id=090819f8b1a8447ca66bc3974d538dfb&scope.project.id=21b65aad6f794b3588a867ef02e946ac&role.id=a4213d713b6f49d78d7bbb9484c9312f => generated 261 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:28.762780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398ba67-e7af-4237-bb26-8a1bcbb26cd7 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:28.767441 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:28.853815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7f7e1-62a7-4efd-8638-088ccffc5fee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:28.859096 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca7f7e1-62a7-4efd-8638-088ccffc5fee None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:28.909348 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:28.927465 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f398ba67-e7af-4237-bb26-8a1bcbb26cd7 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:28.980650 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 196/379] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:35:28 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 519 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:35:28.987992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7f7e1-62a7-4efd-8638-088ccffc5fee None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:28.999268 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab15fb-0140-4430-b473-04d7c17d48bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:28.999921 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab15fb-0140-4430-b473-04d7c17d48bd None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:29.000832 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab15fb-0140-4430-b473-04d7c17d48bd None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:29.001121 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ab15fb-0140-4430-b473-04d7c17d48bd None admin] PATH_INFO: `/v3/roles/member` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:29.012349 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ab15fb-0140-4430-b473-04d7c17d48bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:29.018025 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-14ab15fb-0140-4430-b473-04d7c17d48bd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:35:29.018025 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:29.018025 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:29.018025 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:29.018025 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:29.018025 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:29.018025 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:29.018025 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:29.018025 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:29.018025 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:29.018025 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:29.018025 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:29.018025 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:29.018025 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:29.018025 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:29.018025 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:29.018936 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:29.018936 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:29.018936 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:29.018936 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:29.018936 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:29.018936 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:29.018936 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:29.018936 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:29.018936 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:29.018936 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:29.018936 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:29.018936 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:29.018936 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:29.018936 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:29.018936 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:29.018936 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:29.019771 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:29.019771 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:29.019771 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:29.019771 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:29.019771 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:29.019771 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:29.019771 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:29.019771 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:29.019771 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:29.019771 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:29.019771 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:35:29.019771 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:29.019771 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 184/380] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:35:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:29.025197 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab15fb-0140-4430-b473-04d7c17d48bd None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:29.095997 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:29.102483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca7f7e1-62a7-4efd-8638-088ccffc5fee None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:29.140576 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ab15fb-0140-4430-b473-04d7c17d48bd None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:29.145211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d41aac-c0a2-482f-8232-4ac6f778b046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:29.145869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d41aac-c0a2-482f-8232-4ac6f778b046 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:29.146144 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d41aac-c0a2-482f-8232-4ac6f778b046 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:29.146413 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d41aac-c0a2-482f-8232-4ac6f778b046 None admin] PATH_INFO: `/v3/projects/b86f954a6cac457bad0b226e93315c59` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:29.147606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d41aac-c0a2-482f-8232-4ac6f778b046 None admin] RBAC: Authorizing `identity:get_project(project_id=b86f954a6cac457bad0b226e93315c59)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:29.149159 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d41aac-c0a2-482f-8232-4ac6f778b046 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:29.151295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 197/381] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:35:28 2022] GET /identity/v3/projects/b86f954a6cac457bad0b226e93315c59 => generated 345 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:29.177017 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d41aac-c0a2-482f-8232-4ac6f778b046 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:29.182500 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a655546c-c1d3-4821-9fbb-c6d80b070bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:29.183158 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a655546c-c1d3-4821-9fbb-c6d80b070bbc None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:29.183444 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a655546c-c1d3-4821-9fbb-c6d80b070bbc None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:29.183716 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a655546c-c1d3-4821-9fbb-c6d80b070bbc None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:29.184440 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a655546c-c1d3-4821-9fbb-c6d80b070bbc None admin] RBAC: Adding query filter params (name=member) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:29.184743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a655546c-c1d3-4821-9fbb-c6d80b070bbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:29.186120 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a655546c-c1d3-4821-9fbb-c6d80b070bbc None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:29.196340 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 185/382] 10.222.0.115 () {56 vars in 1106 bytes} [Tue Aug 2 10:35:29 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:35:29.199501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:29.201951 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a655546c-c1d3-4821-9fbb-c6d80b070bbc None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:29.341363 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a655546c-c1d3-4821-9fbb-c6d80b070bbc None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:29.384215 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d41aac-c0a2-482f-8232-4ac6f778b046 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:29.395507 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:29.411630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a328fa-ec6d-4f00-9afb-87bcb8a0e47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:29.412501 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a328fa-ec6d-4f00-9afb-87bcb8a0e47d None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:29.412786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a328fa-ec6d-4f00-9afb-87bcb8a0e47d None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:29.413059 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a328fa-ec6d-4f00-9afb-87bcb8a0e47d None admin] PATH_INFO: `/v3/groups/090819f8b1a8447ca66bc3974d538dfb` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:29.414222 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a328fa-ec6d-4f00-9afb-87bcb8a0e47d None admin] RBAC: Authorizing `identity:get_group(group_id=090819f8b1a8447ca66bc3974d538dfb)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:29.415850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a328fa-ec6d-4f00-9afb-87bcb8a0e47d None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:29.418643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 186/383] 10.222.0.115 () {56 vars in 1185 bytes} [Tue Aug 2 10:35:29 2022] GET /identity/v3/groups/090819f8b1a8447ca66bc3974d538dfb => generated 227 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:29.425346 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a328fa-ec6d-4f00-9afb-87bcb8a0e47d None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:29.461854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5cbf64-dd20-425a-96d8-bf07df723671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:29.461854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cbf64-dd20-425a-96d8-bf07df723671 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:29.461854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cbf64-dd20-425a-96d8-bf07df723671 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:29.461854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5cbf64-dd20-425a-96d8-bf07df723671 None admin] PATH_INFO: `/v3/groups/fc14f4d6ca474719a326c7ff8596c2c9` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:29.467664 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5cbf64-dd20-425a-96d8-bf07df723671 None admin] RBAC: Authorizing `identity:get_group(group_id=fc14f4d6ca474719a326c7ff8596c2c9)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:29.469959 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5cbf64-dd20-425a-96d8-bf07df723671 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:29.472107 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 198/384] 10.222.0.115 () {56 vars in 1185 bytes} [Tue Aug 2 10:35:29 2022] GET /identity/v3/groups/fc14f4d6ca474719a326c7ff8596c2c9 => generated 230 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:35:29.478170 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5cbf64-dd20-425a-96d8-bf07df723671 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:29.571756 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:29.598172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5cbf64-dd20-425a-96d8-bf07df723671 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:29.639747 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98635583-08e2-4eb8-9644-2300f6b7510e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:29.640380 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98635583-08e2-4eb8-9644-2300f6b7510e None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:29.640661 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98635583-08e2-4eb8-9644-2300f6b7510e None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:29.641005 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98635583-08e2-4eb8-9644-2300f6b7510e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:29.642443 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98635583-08e2-4eb8-9644-2300f6b7510e None admin] RBAC: Adding query filter params (group.id=fc14f4d6ca474719a326c7ff8596c2c9, role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac, scope.project.id=b86f954a6cac457bad0b226e93315c59) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:29.642757 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98635583-08e2-4eb8-9644-2300f6b7510e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:29.644305 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98635583-08e2-4eb8-9644-2300f6b7510e None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:29.651624 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 199/385] 10.222.0.115 () {56 vars in 1381 bytes} [Tue Aug 2 10:35:29 2022] GET /identity/v3/role_assignments?group.id=fc14f4d6ca474719a326c7ff8596c2c9&scope.project.id=b86f954a6cac457bad0b226e93315c59&role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 261 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:35:29.680847 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a328fa-ec6d-4f00-9afb-87bcb8a0e47d None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:29.739955 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd56dfe9-164f-4b24-9d93-9b66047c8d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:29.740632 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56dfe9-164f-4b24-9d93-9b66047c8d97 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:29.740907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56dfe9-164f-4b24-9d93-9b66047c8d97 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:29.741179 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd56dfe9-164f-4b24-9d93-9b66047c8d97 None admin] PATH_INFO: `/v3/projects/459ff8f521284cf8b3ec858bfb1ba630` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:29.742528 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd56dfe9-164f-4b24-9d93-9b66047c8d97 None admin] RBAC: Authorizing `identity:get_project(project_id=459ff8f521284cf8b3ec858bfb1ba630)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:29.744040 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd56dfe9-164f-4b24-9d93-9b66047c8d97 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:29.745813 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 187/386] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:35:29 2022] GET /identity/v3/projects/459ff8f521284cf8b3ec858bfb1ba630 => generated 299 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:29.752294 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98635583-08e2-4eb8-9644-2300f6b7510e None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:29.760745 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3926297-7d8a-450c-a9b1-056721328589 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:29.761060 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3926297-7d8a-450c-a9b1-056721328589 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:29.761307 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3926297-7d8a-450c-a9b1-056721328589 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:29.762802 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 188/387] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:35:29.768632 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af63006-9e05-4e55-bc66-849b12550ea6 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:29.768919 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af63006-9e05-4e55-bc66-849b12550ea6 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:29.769194 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af63006-9e05-4e55-bc66-849b12550ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:29.808186 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-1af63006-9e05-4e55-bc66-849b12550ea6 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:29.809432 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-1af63006-9e05-4e55-bc66-849b12550ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:29.905051 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98635583-08e2-4eb8-9644-2300f6b7510e None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:29.939657 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:29.944936 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce96f36-8da4-4803-b736-b445e572ae8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:29.945616 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce96f36-8da4-4803-b736-b445e572ae8d None admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:29.945891 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce96f36-8da4-4803-b736-b445e572ae8d None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:29.946162 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce96f36-8da4-4803-b736-b445e572ae8d None admin] PATH_INFO: `/v3/projects/459ff8f521284cf8b3ec858bfb1ba630/groups/090819f8b1a8447ca66bc3974d538dfb/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:29.948469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce96f36-8da4-4803-b736-b445e572ae8d None admin] RBAC: Authorizing `identity:create_grant(project_id=459ff8f521284cf8b3ec858bfb1ba630, group_id=090819f8b1a8447ca66bc3974d538dfb, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:29.949944 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce96f36-8da4-4803-b736-b445e572ae8d None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:29.964837 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 189/388] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:29 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:35:29.992131 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af63006-9e05-4e55-bc66-849b12550ea6 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:30.057646 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:30.065517 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 200/389] 10.222.0.115 () {58 vars in 1447 bytes} [Tue Aug 2 10:35:29 2022] PUT /identity/v3/projects/459ff8f521284cf8b3ec858bfb1ba630/groups/090819f8b1a8447ca66bc3974d538dfb/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:35:30.113958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af63006-9e05-4e55-bc66-849b12550ea6 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:30.152615 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6b30f-a96a-484a-a06f-d2d12ee221c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:30.153350 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6b30f-a96a-484a-a06f-d2d12ee221c0 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:30.153544 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6b30f-a96a-484a-a06f-d2d12ee221c0 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:30.153793 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6b30f-a96a-484a-a06f-d2d12ee221c0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:30.166538 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc6b30f-a96a-484a-a06f-d2d12ee221c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:30.172031 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-0dc6b30f-a96a-484a-a06f-d2d12ee221c0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:35:30.172031 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:30.172031 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:30.172031 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:30.172031 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:30.172031 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:30.172031 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:30.172031 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:30.172031 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:30.172031 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:30.172031 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:30.172031 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:30.172031 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:30.172031 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:30.172031 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:30.172031 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:30.172994 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:30.172994 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:30.172994 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:30.172994 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:30.172994 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:30.172994 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:30.172994 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:30.172994 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:30.172994 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:30.172994 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:30.172994 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:30.172994 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:30.172994 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:30.172994 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:30.172994 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:30.172994 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:30.173819 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:30.173819 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:30.173819 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:30.173819 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:30.173819 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:30.173819 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:30.173819 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:30.173819 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:30.173819 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:30.173819 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:30.173819 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:35:30.173819 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:30.173819 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 190/390] 10.222.0.115 () {56 vars in 1119 bytes} [Tue Aug 2 10:35:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:35:30.179969 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce96f36-8da4-4803-b736-b445e572ae8d None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:30.315811 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce96f36-8da4-4803-b736-b445e572ae8d None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:30.369239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d1523-a4ba-4ba3-969d-0f252b3e933e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:30.369873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d1523-a4ba-4ba3-969d-0f252b3e933e None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:30.370156 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d1523-a4ba-4ba3-969d-0f252b3e933e None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:30.370433 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d1523-a4ba-4ba3-969d-0f252b3e933e None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:30.371189 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d1523-a4ba-4ba3-969d-0f252b3e933e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:30.371500 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d1523-a4ba-4ba3-969d-0f252b3e933e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:30.372875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d1523-a4ba-4ba3-969d-0f252b3e933e None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:30.382402 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 201/391] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:35:30 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:30.391773 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6b30f-a96a-484a-a06f-d2d12ee221c0 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:30.443632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8ef6f-2a51-4450-90c0-f3dfdfb90ce8 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:30.443922 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8ef6f-2a51-4450-90c0-f3dfdfb90ce8 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:30.444141 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8ef6f-2a51-4450-90c0-f3dfdfb90ce8 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:30.445598 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 202/392] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:30.451482 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d221220-6a83-4b68-9984-ee5fa0172540 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:30.451774 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d221220-6a83-4b68-9984-ee5fa0172540 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:30.452057 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d221220-6a83-4b68-9984-ee5fa0172540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:30.459349 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:30.488116 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-9d221220-6a83-4b68-9984-ee5fa0172540 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:30.489358 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-9d221220-6a83-4b68-9984-ee5fa0172540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:30.526474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6b30f-a96a-484a-a06f-d2d12ee221c0 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:30.573782 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1418c57-fe7c-4dfd-b717-cb53da95e920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:30.574436 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1418c57-fe7c-4dfd-b717-cb53da95e920 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:30.574711 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1418c57-fe7c-4dfd-b717-cb53da95e920 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:30.574979 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1418c57-fe7c-4dfd-b717-cb53da95e920 None admin] PATH_INFO: `/v3/groups/090819f8b1a8447ca66bc3974d538dfb` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:30.577007 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1418c57-fe7c-4dfd-b717-cb53da95e920 None admin] RBAC: Authorizing `identity:get_group(group_id=090819f8b1a8447ca66bc3974d538dfb)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:30.582225 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1418c57-fe7c-4dfd-b717-cb53da95e920 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:30.585433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 191/393] 10.222.0.115 () {56 vars in 1185 bytes} [Tue Aug 2 10:35:30 2022] GET /identity/v3/groups/090819f8b1a8447ca66bc3974d538dfb => generated 227 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:30.586920 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:30.591384 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1418c57-fe7c-4dfd-b717-cb53da95e920 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:30.612394 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 203/394] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:30 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:35:30.638985 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d221220-6a83-4b68-9984-ee5fa0172540 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:30.780952 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1418c57-fe7c-4dfd-b717-cb53da95e920 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:30.800150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d221220-6a83-4b68-9984-ee5fa0172540 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:30.823491 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:30.836625 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877856e-4679-4401-8420-da91fc8972f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:30.837570 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877856e-4679-4401-8420-da91fc8972f9 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:30.842011 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877856e-4679-4401-8420-da91fc8972f9 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:30.842011 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877856e-4679-4401-8420-da91fc8972f9 None admin] PATH_INFO: `/v3/projects/459ff8f521284cf8b3ec858bfb1ba630` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:30.842011 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877856e-4679-4401-8420-da91fc8972f9 None admin] RBAC: Authorizing `identity:get_project(project_id=459ff8f521284cf8b3ec858bfb1ba630)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:30.842011 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877856e-4679-4401-8420-da91fc8972f9 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:30.842339 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c591491e-733a-498e-996f-f17dc2aeacf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:30.842339 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591491e-733a-498e-996f-f17dc2aeacf9 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:30.842339 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591491e-733a-498e-996f-f17dc2aeacf9 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:30.842596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c591491e-733a-498e-996f-f17dc2aeacf9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:30.845191 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 192/395] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:35:30 2022] GET /identity/v3/projects/459ff8f521284cf8b3ec858bfb1ba630 => generated 299 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:30.853605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877856e-4679-4401-8420-da91fc8972f9 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:30.855319 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c591491e-733a-498e-996f-f17dc2aeacf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:30.860530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-c591491e-733a-498e-996f-f17dc2aeacf9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:35:30.860530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:30.860530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:30.860530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:30.860530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:30.860530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:30.860530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:30.860530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:30.860530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:30.860530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:30.860530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:30.860530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:30.860530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:30.860530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:30.860530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:30.860530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:30.861624 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:30.861624 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:30.861624 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:30.861624 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:30.861624 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:30.861624 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:30.861624 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:30.861624 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:30.861624 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:30.861624 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:30.861624 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:30.861624 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:30.861624 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:30.861624 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:30.861624 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:30.861624 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:30.862450 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:30.862450 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:30.862450 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:30.862450 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:30.862450 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:30.862450 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:30.862450 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:30.862450 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:30.862450 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:30.862450 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:30.862450 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:35:30.862450 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:30.862450 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 204/396] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:35:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:30.878723 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c591491e-733a-498e-996f-f17dc2aeacf9 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:30.944325 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:30.979084 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877856e-4679-4401-8420-da91fc8972f9 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:31.010218 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c591491e-733a-498e-996f-f17dc2aeacf9 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:31.027781 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64b27d-6146-4bc9-a283-949eeb96f1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:31.028460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64b27d-6146-4bc9-a283-949eeb96f1f4 None admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:31.028731 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64b27d-6146-4bc9-a283-949eeb96f1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:31.029000 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc64b27d-6146-4bc9-a283-949eeb96f1f4 None admin] PATH_INFO: `/v3/projects/459ff8f521284cf8b3ec858bfb1ba630/groups/090819f8b1a8447ca66bc3974d538dfb/roles/a4213d713b6f49d78d7bbb9484c9312f` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:31.035397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc64b27d-6146-4bc9-a283-949eeb96f1f4 None admin] RBAC: Authorizing `identity:create_grant(project_id=459ff8f521284cf8b3ec858bfb1ba630, group_id=090819f8b1a8447ca66bc3974d538dfb, role_id=a4213d713b6f49d78d7bbb9484c9312f)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:31.036846 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc64b27d-6146-4bc9-a283-949eeb96f1f4 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:31.066402 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb563634-8cc9-4fa7-9cdd-8bfed3f1322a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:31.067083 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb563634-8cc9-4fa7-9cdd-8bfed3f1322a None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:31.067372 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb563634-8cc9-4fa7-9cdd-8bfed3f1322a None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:31.067648 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb563634-8cc9-4fa7-9cdd-8bfed3f1322a None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:31.068436 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb563634-8cc9-4fa7-9cdd-8bfed3f1322a None admin] RBAC: Adding query filter params (name=member) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:31.068747 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb563634-8cc9-4fa7-9cdd-8bfed3f1322a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:31.070142 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb563634-8cc9-4fa7-9cdd-8bfed3f1322a None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:31.079719 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 205/397] 10.222.0.115 () {56 vars in 1106 bytes} [Tue Aug 2 10:35:30 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:31.085844 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb563634-8cc9-4fa7-9cdd-8bfed3f1322a None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:31.129646 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 193/398] 10.222.0.115 () {58 vars in 1447 bytes} [Tue Aug 2 10:35:30 2022] PUT /identity/v3/projects/459ff8f521284cf8b3ec858bfb1ba630/groups/090819f8b1a8447ca66bc3974d538dfb/roles/a4213d713b6f49d78d7bbb9484c9312f => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:35:31.203792 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb563634-8cc9-4fa7-9cdd-8bfed3f1322a None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:31.222433 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:31.253505 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8376f1a-85f9-46fe-9155-36cd3239e4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:31.254558 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8376f1a-85f9-46fe-9155-36cd3239e4ac None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:31.255071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8376f1a-85f9-46fe-9155-36cd3239e4ac None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:31.255553 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8376f1a-85f9-46fe-9155-36cd3239e4ac None admin] PATH_INFO: `/v3/groups/090819f8b1a8447ca66bc3974d538dfb` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:31.259824 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8376f1a-85f9-46fe-9155-36cd3239e4ac None admin] RBAC: Authorizing `identity:get_group(group_id=090819f8b1a8447ca66bc3974d538dfb)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:31.262077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8376f1a-85f9-46fe-9155-36cd3239e4ac None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:31.264154 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 206/399] 10.222.0.115 () {56 vars in 1185 bytes} [Tue Aug 2 10:35:31 2022] GET /identity/v3/groups/090819f8b1a8447ca66bc3974d538dfb => generated 227 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:31.270713 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64b27d-6146-4bc9-a283-949eeb96f1f4 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:31.365586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:31.443430 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc64b27d-6146-4bc9-a283-949eeb96f1f4 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:31.492728 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86f58eb-f62f-4c26-8aab-d3ae1214d2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:31.493417 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f58eb-f62f-4c26-8aab-d3ae1214d2f0 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:31.493698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f58eb-f62f-4c26-8aab-d3ae1214d2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:31.493968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86f58eb-f62f-4c26-8aab-d3ae1214d2f0 None admin] PATH_INFO: `/v3/projects/21b65aad6f794b3588a867ef02e946ac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:31.497267 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f58eb-f62f-4c26-8aab-d3ae1214d2f0 None admin] RBAC: Authorizing `identity:get_project(project_id=21b65aad6f794b3588a867ef02e946ac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:31.497267 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86f58eb-f62f-4c26-8aab-d3ae1214d2f0 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:31.498809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 194/400] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:35:31 2022] GET /identity/v3/projects/21b65aad6f794b3588a867ef02e946ac => generated 303 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:31.505311 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8376f1a-85f9-46fe-9155-36cd3239e4ac None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:31.566402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb665fb2-1c47-44d2-97bf-3add09e89686 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:31.566897 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb665fb2-1c47-44d2-97bf-3add09e89686 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:31.567482 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb665fb2-1c47-44d2-97bf-3add09e89686 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:31.569849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 195/401] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:31 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:31.576017 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5e719-4736-4ba7-bd82-de129c943d46 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:31.576459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5e719-4736-4ba7-bd82-de129c943d46 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:31.576936 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5e719-4736-4ba7-bd82-de129c943d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:31.610710 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-36f5e719-4736-4ba7-bd82-de129c943d46 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:31.612473 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-36f5e719-4736-4ba7-bd82-de129c943d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:31.678847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8376f1a-85f9-46fe-9155-36cd3239e4ac None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:31.724197 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d355a7-50c6-4ce0-a23b-77acacd2d5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:31.724870 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d355a7-50c6-4ce0-a23b-77acacd2d5f2 None admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:31.725146 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d355a7-50c6-4ce0-a23b-77acacd2d5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:31.725974 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d355a7-50c6-4ce0-a23b-77acacd2d5f2 None admin] PATH_INFO: `/v3/projects/21b65aad6f794b3588a867ef02e946ac/groups/090819f8b1a8447ca66bc3974d538dfb/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:31.728611 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d355a7-50c6-4ce0-a23b-77acacd2d5f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=21b65aad6f794b3588a867ef02e946ac, group_id=090819f8b1a8447ca66bc3974d538dfb, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:31.730055 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d355a7-50c6-4ce0-a23b-77acacd2d5f2 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:31.763243 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 196/402] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:31 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:31.788334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5e719-4736-4ba7-bd82-de129c943d46 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:31.805619 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 207/403] 10.222.0.115 () {58 vars in 1447 bytes} [Tue Aug 2 10:35:31 2022] PUT /identity/v3/projects/21b65aad6f794b3588a867ef02e946ac/groups/090819f8b1a8447ca66bc3974d538dfb/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:35:31.862390 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:31.944460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5e719-4736-4ba7-bd82-de129c943d46 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:32.003380 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f0699-c84d-4846-b1f1-98d66890908e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:32.004054 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f0699-c84d-4846-b1f1-98d66890908e None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:32.004334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f0699-c84d-4846-b1f1-98d66890908e None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:32.004611 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0f0699-c84d-4846-b1f1-98d66890908e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:32.023369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0f0699-c84d-4846-b1f1-98d66890908e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:32.028707 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-9b0f0699-c84d-4846-b1f1-98d66890908e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:35:32.028707 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:32.028707 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:32.028707 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:32.028707 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:32.028707 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:32.028707 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:32.028707 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:32.028707 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:32.028707 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:32.028707 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:32.028707 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:32.028707 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:32.028707 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:32.028707 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:32.028707 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:32.029669 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:32.029669 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:32.029669 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:32.029669 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:32.029669 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:32.029669 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:32.029669 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:32.029669 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:32.029669 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:32.029669 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:32.029669 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:32.029669 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:32.029669 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:32.029669 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:32.029669 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:32.029669 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:32.030472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:32.030472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:32.030472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:32.030472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:32.030472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:32.030472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:32.030472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:32.030472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:32.030472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:32.030472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:32.030472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:35:32.030472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:32.030472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 197/404] 10.222.0.115 () {56 vars in 1119 bytes} [Tue Aug 2 10:35:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:35:32.036176 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d355a7-50c6-4ce0-a23b-77acacd2d5f2 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:32.089876 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:32.153228 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d355a7-50c6-4ce0-a23b-77acacd2d5f2 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:32.191664 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c17dc8-772c-4867-81d3-80968d3d3726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:32.192636 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c17dc8-772c-4867-81d3-80968d3d3726 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:32.193134 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c17dc8-772c-4867-81d3-80968d3d3726 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:32.193607 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c17dc8-772c-4867-81d3-80968d3d3726 None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:32.194708 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c17dc8-772c-4867-81d3-80968d3d3726 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:32.195292 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c17dc8-772c-4867-81d3-80968d3d3726 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:32.196854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c17dc8-772c-4867-81d3-80968d3d3726 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:32.207814 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 208/405] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:35:32 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:32.213563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f0699-c84d-4846-b1f1-98d66890908e None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:32.359193 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0f0699-c84d-4846-b1f1-98d66890908e None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:32.361706 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:32.396035 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03647d8-9152-4625-8716-d014279c2f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:32.396689 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03647d8-9152-4625-8716-d014279c2f14 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:32.396970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03647d8-9152-4625-8716-d014279c2f14 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:32.397244 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03647d8-9152-4625-8716-d014279c2f14 None admin] PATH_INFO: `/v3/groups/090819f8b1a8447ca66bc3974d538dfb` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:32.398413 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03647d8-9152-4625-8716-d014279c2f14 None admin] RBAC: Authorizing `identity:get_group(group_id=090819f8b1a8447ca66bc3974d538dfb)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:32.399883 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03647d8-9152-4625-8716-d014279c2f14 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:32.401852 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 198/406] 10.222.0.115 () {56 vars in 1185 bytes} [Tue Aug 2 10:35:32 2022] GET /identity/v3/groups/090819f8b1a8447ca66bc3974d538dfb => generated 227 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:32.408003 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c17dc8-772c-4867-81d3-80968d3d3726 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:32.441932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9815d19-42a8-49c0-9a6e-623bf49490f1 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:32.441932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9815d19-42a8-49c0-9a6e-623bf49490f1 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:32.441932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9815d19-42a8-49c0-9a6e-623bf49490f1 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:32.443394 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 199/407] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:35:32.451138 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cdb48-a565-4e05-b353-7717ee1c6f1a None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:32.451138 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cdb48-a565-4e05-b353-7717ee1c6f1a None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:32.451480 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43cdb48-a565-4e05-b353-7717ee1c6f1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:32.496139 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-f43cdb48-a565-4e05-b353-7717ee1c6f1a None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:32.497424 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f43cdb48-a565-4e05-b353-7717ee1c6f1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:32.551082 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c17dc8-772c-4867-81d3-80968d3d3726 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:32.553121 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:32.589999 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5dc9f-c7b2-4bc9-88bd-bd9f50a6fc1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:32.591231 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5dc9f-c7b2-4bc9-88bd-bd9f50a6fc1b None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:32.591826 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5dc9f-c7b2-4bc9-88bd-bd9f50a6fc1b None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:32.592420 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c5dc9f-c7b2-4bc9-88bd-bd9f50a6fc1b None admin] PATH_INFO: `/v3/projects/21b65aad6f794b3588a867ef02e946ac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:32.594539 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5dc9f-c7b2-4bc9-88bd-bd9f50a6fc1b None admin] RBAC: Authorizing `identity:get_project(project_id=21b65aad6f794b3588a867ef02e946ac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:32.596751 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c5dc9f-c7b2-4bc9-88bd-bd9f50a6fc1b None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:32.599014 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 209/408] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:35:32 2022] GET /identity/v3/projects/21b65aad6f794b3588a867ef02e946ac => generated 303 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:32.609585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5dc9f-c7b2-4bc9-88bd-bd9f50a6fc1b None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:32.645820 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 200/409] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:32 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:35:32.692158 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43cdb48-a565-4e05-b353-7717ee1c6f1a None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:32.742353 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c5dc9f-c7b2-4bc9-88bd-bd9f50a6fc1b None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:32.779030 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d98088-b6a1-4cc1-94a9-aaeef8b6c225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:32.780142 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d98088-b6a1-4cc1-94a9-aaeef8b6c225 None admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:32.780634 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d98088-b6a1-4cc1-94a9-aaeef8b6c225 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:32.781099 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d98088-b6a1-4cc1-94a9-aaeef8b6c225 None admin] PATH_INFO: `/v3/projects/21b65aad6f794b3588a867ef02e946ac/groups/090819f8b1a8447ca66bc3974d538dfb/roles/a4213d713b6f49d78d7bbb9484c9312f` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:32.784613 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d98088-b6a1-4cc1-94a9-aaeef8b6c225 None admin] RBAC: Authorizing `identity:create_grant(project_id=21b65aad6f794b3588a867ef02e946ac, group_id=090819f8b1a8447ca66bc3974d538dfb, role_id=a4213d713b6f49d78d7bbb9484c9312f)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:32.786317 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d98088-b6a1-4cc1-94a9-aaeef8b6c225 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:32.843906 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43cdb48-a565-4e05-b353-7717ee1c6f1a None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:32.846826 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:32.856971 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 210/410] 10.222.0.115 () {58 vars in 1447 bytes} [Tue Aug 2 10:35:32 2022] PUT /identity/v3/projects/21b65aad6f794b3588a867ef02e946ac/groups/090819f8b1a8447ca66bc3974d538dfb/roles/a4213d713b6f49d78d7bbb9484c9312f => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:35:32.895100 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93326c34-1cc0-40df-8f70-14a00c221244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:32.895752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93326c34-1cc0-40df-8f70-14a00c221244 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:32.896030 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93326c34-1cc0-40df-8f70-14a00c221244 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:32.896301 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93326c34-1cc0-40df-8f70-14a00c221244 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:32.909201 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93326c34-1cc0-40df-8f70-14a00c221244 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:32.914576 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-93326c34-1cc0-40df-8f70-14a00c221244 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:35:32.914576 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:32.914576 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:32.914576 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:32.914576 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:32.914576 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:32.914576 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:32.914576 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:32.914576 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:32.914576 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:32.914576 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:32.914576 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:32.914576 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:32.914576 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:32.914576 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:32.914576 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:32.915787 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:32.915787 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:32.915787 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:32.915787 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:32.915787 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:32.915787 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:32.915787 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:32.915787 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:32.915787 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:32.915787 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:32.915787 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:32.915787 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:32.915787 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:32.915787 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:32.915787 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:32.915787 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:32.916596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:32.916596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:32.916596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:32.916596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:32.916596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:32.916596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:32.916596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:32.916596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:32.916596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:32.916596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:32.916596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:35:32.916596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:32.916596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 201/411] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:35:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:35:32.924330 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d98088-b6a1-4cc1-94a9-aaeef8b6c225 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:32.959885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:33.047615 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d98088-b6a1-4cc1-94a9-aaeef8b6c225 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:33.086696 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcd5019-bdfa-4b56-98d4-9e5fd8c73d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:33.087437 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd5019-bdfa-4b56-98d4-9e5fd8c73d5e None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:33.087716 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd5019-bdfa-4b56-98d4-9e5fd8c73d5e None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:33.087988 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcd5019-bdfa-4b56-98d4-9e5fd8c73d5e None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:33.088771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd5019-bdfa-4b56-98d4-9e5fd8c73d5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:33.089074 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd5019-bdfa-4b56-98d4-9e5fd8c73d5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:33.090515 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcd5019-bdfa-4b56-98d4-9e5fd8c73d5e None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:33.104846 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 211/412] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:35:32 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:33.111856 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93326c34-1cc0-40df-8f70-14a00c221244 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:33.222927 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93326c34-1cc0-40df-8f70-14a00c221244 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:33.256031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb2fbf-d864-4bf1-9cc2-f60d4616b7d9 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:33.256687 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb2fbf-d864-4bf1-9cc2-f60d4616b7d9 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:33.257194 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb2fbf-d864-4bf1-9cc2-f60d4616b7d9 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:33.259731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 212/413] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:33 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:35:33.261675 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e62ca5-6b33-4efd-a383-82c82c1a7802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:33.262466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e62ca5-6b33-4efd-a383-82c82c1a7802 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:33.262902 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e62ca5-6b33-4efd-a383-82c82c1a7802 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:33.263323 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e62ca5-6b33-4efd-a383-82c82c1a7802 None admin] PATH_INFO: `/v3/groups/fc14f4d6ca474719a326c7ff8596c2c9` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:33.265080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e62ca5-6b33-4efd-a383-82c82c1a7802 None admin] RBAC: Authorizing `identity:get_group(group_id=fc14f4d6ca474719a326c7ff8596c2c9)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:33.265989 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2c8e5-db82-4f0f-aed2-b2fee4b65c1d None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:33.265989 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2c8e5-db82-4f0f-aed2-b2fee4b65c1d None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:33.266441 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2c8e5-db82-4f0f-aed2-b2fee4b65c1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:33.267701 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e62ca5-6b33-4efd-a383-82c82c1a7802 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:33.269878 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 202/414] 10.222.0.115 () {56 vars in 1185 bytes} [Tue Aug 2 10:35:33 2022] GET /identity/v3/groups/fc14f4d6ca474719a326c7ff8596c2c9 => generated 230 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:33.275369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e62ca5-6b33-4efd-a383-82c82c1a7802 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:33.309006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-59e2c8e5-db82-4f0f-aed2-b2fee4b65c1d None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:33.310147 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-59e2c8e5-db82-4f0f-aed2-b2fee4b65c1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:33.346130 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:33.452769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 213/415] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:33 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:35:33.467004 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e62ca5-6b33-4efd-a383-82c82c1a7802 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:33.502188 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2c8e5-db82-4f0f-aed2-b2fee4b65c1d None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:33.540009 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:33.548995 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0453ca-4ce0-4321-a8ec-3cdd157eb7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:33.549659 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0453ca-4ce0-4321-a8ec-3cdd157eb7d1 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:33.549943 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0453ca-4ce0-4321-a8ec-3cdd157eb7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:33.550221 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0453ca-4ce0-4321-a8ec-3cdd157eb7d1 None admin] PATH_INFO: `/v3/projects/b86f954a6cac457bad0b226e93315c59` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:33.555797 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0453ca-4ce0-4321-a8ec-3cdd157eb7d1 None admin] RBAC: Authorizing `identity:get_project(project_id=b86f954a6cac457bad0b226e93315c59)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:33.557300 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0453ca-4ce0-4321-a8ec-3cdd157eb7d1 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:33.566406 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 203/416] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:35:33 2022] GET /identity/v3/projects/b86f954a6cac457bad0b226e93315c59 => generated 345 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:33.570884 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0453ca-4ce0-4321-a8ec-3cdd157eb7d1 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:33.623170 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2c8e5-db82-4f0f-aed2-b2fee4b65c1d None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:33.663395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecea9c-7291-4773-8459-59a7f95d194f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:33.664391 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecea9c-7291-4773-8459-59a7f95d194f None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:33.664888 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecea9c-7291-4773-8459-59a7f95d194f None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:33.665363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ecea9c-7291-4773-8459-59a7f95d194f None admin] PATH_INFO: `/v3/roles/member` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:33.677301 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ecea9c-7291-4773-8459-59a7f95d194f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:33.682831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-40ecea9c-7291-4773-8459-59a7f95d194f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:35:33.682831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:33.682831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:33.682831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:33.682831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:33.682831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:33.682831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:33.682831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:33.682831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:33.682831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:33.682831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:33.682831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:33.682831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:33.682831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:33.682831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:33.682831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:33.683908 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:33.683908 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:33.683908 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:33.683908 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:33.683908 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:33.683908 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:33.683908 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:33.683908 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:33.683908 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:33.683908 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:33.683908 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:33.683908 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:33.683908 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:33.683908 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:33.683908 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:33.683908 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:33.684935 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:33.684935 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:33.684935 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:33.684935 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:33.684935 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:33.684935 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:33.684935 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:33.684935 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:33.684935 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:33.684935 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:33.684935 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:35:33.684935 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:33.691456 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 214/417] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:35:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:35:33.693769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecea9c-7291-4773-8459-59a7f95d194f None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:33.772402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:33.777452 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0453ca-4ce0-4321-a8ec-3cdd157eb7d1 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:33.787484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:33.807947 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ecea9c-7291-4773-8459-59a7f95d194f None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:33.849246 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc81f0-9a31-4e13-bfd8-a1ac7e36295a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:33.849769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da97c5c-ab11-4a18-88dc-015d3c1e46e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:33.849893 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc81f0-9a31-4e13-bfd8-a1ac7e36295a None admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:33.850107 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc81f0-9a31-4e13-bfd8-a1ac7e36295a None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:33.850381 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc81f0-9a31-4e13-bfd8-a1ac7e36295a None admin] PATH_INFO: `/v3/projects/b86f954a6cac457bad0b226e93315c59/groups/fc14f4d6ca474719a326c7ff8596c2c9/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:33.851323 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da97c5c-ab11-4a18-88dc-015d3c1e46e6 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:33.851913 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da97c5c-ab11-4a18-88dc-015d3c1e46e6 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:33.852473 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da97c5c-ab11-4a18-88dc-015d3c1e46e6 None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:33.853368 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da97c5c-ab11-4a18-88dc-015d3c1e46e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:33.853817 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da97c5c-ab11-4a18-88dc-015d3c1e46e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:33.855481 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da97c5c-ab11-4a18-88dc-015d3c1e46e6 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:33.857907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc81f0-9a31-4e13-bfd8-a1ac7e36295a None admin] RBAC: Authorizing `identity:create_grant(project_id=b86f954a6cac457bad0b226e93315c59, group_id=fc14f4d6ca474719a326c7ff8596c2c9, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:33.859932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc81f0-9a31-4e13-bfd8-a1ac7e36295a None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:33.870555 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 215/418] 10.222.0.115 () {56 vars in 1106 bytes} [Tue Aug 2 10:35:33 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:35:33.879300 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da97c5c-ab11-4a18-88dc-015d3c1e46e6 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:33.941217 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 204/419] 10.222.0.115 () {58 vars in 1447 bytes} [Tue Aug 2 10:35:33 2022] PUT /identity/v3/projects/b86f954a6cac457bad0b226e93315c59/groups/fc14f4d6ca474719a326c7ff8596c2c9/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:35:33.995909 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ad21f-667b-41c1-a476-23abc31eed68 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:33.996245 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ad21f-667b-41c1-a476-23abc31eed68 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:33.996522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7ad21f-667b-41c1-a476-23abc31eed68 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:33.998314 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 205/420] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:34.004330 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd9f4d1-3833-42a3-935a-6343bde71c41 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:34.004615 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd9f4d1-3833-42a3-935a-6343bde71c41 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:34.004889 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd9f4d1-3833-42a3-935a-6343bde71c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:34.016434 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da97c5c-ab11-4a18-88dc-015d3c1e46e6 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:34.056232 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-9bd9f4d1-3833-42a3-935a-6343bde71c41 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:34.057460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-9bd9f4d1-3833-42a3-935a-6343bde71c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:34.058618 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a24676-a0a5-45c2-af3e-7efaa203ebdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:34.059594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a24676-a0a5-45c2-af3e-7efaa203ebdd None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:34.060045 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a24676-a0a5-45c2-af3e-7efaa203ebdd None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:34.060474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a24676-a0a5-45c2-af3e-7efaa203ebdd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:34.087761 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:34.143024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a24676-a0a5-45c2-af3e-7efaa203ebdd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:34.144760 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a24676-a0a5-45c2-af3e-7efaa203ebdd None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:34.240795 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 206/421] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:34 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:34.256660 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 216/422] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:35:33 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 10:35:34.262479 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd9f4d1-3833-42a3-935a-6343bde71c41 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:34.269563 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a24676-a0a5-45c2-af3e-7efaa203ebdd None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:34.319990 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:34.380353 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd9f4d1-3833-42a3-935a-6343bde71c41 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:34.383414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:34.383728 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a24676-a0a5-45c2-af3e-7efaa203ebdd None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:34.418935 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197f6ac-28ac-4d65-b3ea-a77c9af61eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:34.419140 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1954e57-dd76-4092-ae2b-7259402195f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:34.419523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197f6ac-28ac-4d65-b3ea-a77c9af61eed None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:34.419801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197f6ac-28ac-4d65-b3ea-a77c9af61eed None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:34.420071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197f6ac-28ac-4d65-b3ea-a77c9af61eed None admin] PATH_INFO: `/v3/projects/459ff8f521284cf8b3ec858bfb1ba630` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:34.420317 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1954e57-dd76-4092-ae2b-7259402195f8 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:34.420875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1954e57-dd76-4092-ae2b-7259402195f8 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:34.421251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1954e57-dd76-4092-ae2b-7259402195f8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:34.421668 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197f6ac-28ac-4d65-b3ea-a77c9af61eed None admin] RBAC: Authorizing `identity:get_project(project_id=459ff8f521284cf8b3ec858bfb1ba630)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:34.424586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197f6ac-28ac-4d65-b3ea-a77c9af61eed None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:34.426703 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 207/423] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:35:34 2022] GET /identity/v3/projects/459ff8f521284cf8b3ec858bfb1ba630 => generated 299 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:34.428022 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d6319-dbcc-48ec-a3e2-a891b9309afc None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:34.428279 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d6319-dbcc-48ec-a3e2-a891b9309afc None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:34.429306 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d6319-dbcc-48ec-a3e2-a891b9309afc None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:34.434091 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 208/424] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:34 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:34.440603 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03accd-5e2e-4505-b07b-99054447d1b6 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:34.440752 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1954e57-dd76-4092-ae2b-7259402195f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:34.441217 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03accd-5e2e-4505-b07b-99054447d1b6 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:34.441696 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc03accd-5e2e-4505-b07b-99054447d1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:34.451251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-b1954e57-dd76-4092-ae2b-7259402195f8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:35:34.451251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:34.451251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:34.451251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:34.451251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:34.451251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:34.451251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:34.451251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:34.451251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:34.451251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:34.451251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:34.451251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:34.451251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:34.451251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:34.451251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:34.451251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:34.452941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:34.452941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:34.452941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:34.452941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:34.452941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:34.452941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:34.452941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:34.452941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:34.452941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:34.452941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:34.452941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:34.452941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:34.452941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:34.452941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:34.452941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:34.452941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:34.454202 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:34.454202 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:34.454202 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:34.454202 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:34.454202 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:34.454202 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:34.454202 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:34.454202 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:34.454202 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:34.454202 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:34.454202 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:35:34.454202 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:34.459609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 217/425] 10.222.0.115 () {56 vars in 1119 bytes} [Tue Aug 2 10:35:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:35:34.459609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1954e57-dd76-4092-ae2b-7259402195f8 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:34.479292 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-dc03accd-5e2e-4505-b07b-99054447d1b6 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:34.480703 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-dc03accd-5e2e-4505-b07b-99054447d1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:34.567326 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 209/426] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:34 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:34.568269 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03accd-5e2e-4505-b07b-99054447d1b6 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:34.588298 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1954e57-dd76-4092-ae2b-7259402195f8 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:34.592197 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:34.631716 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0d69d-c5db-46a3-98c7-2d17403d7431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:34.632373 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0d69d-c5db-46a3-98c7-2d17403d7431 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:34.632653 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0d69d-c5db-46a3-98c7-2d17403d7431 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:34.632931 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add0d69d-c5db-46a3-98c7-2d17403d7431 None admin] PATH_INFO: `/v3/groups/090819f8b1a8447ca66bc3974d538dfb` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:34.634348 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add0d69d-c5db-46a3-98c7-2d17403d7431 None admin] RBAC: Authorizing `identity:get_group(group_id=090819f8b1a8447ca66bc3974d538dfb)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:34.635680 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add0d69d-c5db-46a3-98c7-2d17403d7431 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:34.637700 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 218/427] 10.222.0.115 () {56 vars in 1185 bytes} [Tue Aug 2 10:35:34 2022] GET /identity/v3/groups/090819f8b1a8447ca66bc3974d538dfb => generated 227 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:34.638623 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0d69d-c5db-46a3-98c7-2d17403d7431 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:34.701214 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:34.719869 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc03accd-5e2e-4505-b07b-99054447d1b6 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:34.757561 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add0d69d-c5db-46a3-98c7-2d17403d7431 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:34.759013 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af6c31-58a6-4081-a192-35eb61d09eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:34.759935 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af6c31-58a6-4081-a192-35eb61d09eb1 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:34.760413 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af6c31-58a6-4081-a192-35eb61d09eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:34.760866 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af6c31-58a6-4081-a192-35eb61d09eb1 None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:34.761785 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af6c31-58a6-4081-a192-35eb61d09eb1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:34.762257 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af6c31-58a6-4081-a192-35eb61d09eb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:34.763822 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af6c31-58a6-4081-a192-35eb61d09eb1 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:34.778735 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 210/428] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:35:34 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:34.779954 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af6c31-58a6-4081-a192-35eb61d09eb1 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:34.795876 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38a7b5-3a13-45ca-8e16-dd28104640ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:34.796534 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38a7b5-3a13-45ca-8e16-dd28104640ed None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:34.796894 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38a7b5-3a13-45ca-8e16-dd28104640ed None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:34.797200 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc38a7b5-3a13-45ca-8e16-dd28104640ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:34.808492 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc38a7b5-3a13-45ca-8e16-dd28104640ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:34.814970 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-dc38a7b5-3a13-45ca-8e16-dd28104640ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:35:34.814970 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:34.814970 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:34.814970 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:34.814970 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:34.814970 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:34.814970 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:34.814970 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:34.814970 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:34.814970 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:34.814970 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:34.814970 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:34.814970 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:34.814970 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:34.814970 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:34.814970 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:34.815879 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:34.815879 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:34.815879 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:34.815879 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:34.815879 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:34.815879 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:34.815879 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:34.815879 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:34.815879 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:34.815879 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:34.815879 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:34.815879 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:34.815879 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:34.815879 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:34.815879 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:34.815879 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:34.817059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:34.817059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:34.817059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:34.817059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:34.817059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:34.817059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:34.817059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:34.817059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:34.817059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:34.817059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:34.817059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 02 10:35:34.817059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:34.820825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 219/429] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:35:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:34.821770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38a7b5-3a13-45ca-8e16-dd28104640ed None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:34.856472 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:34.904294 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af6c31-58a6-4081-a192-35eb61d09eb1 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:34.936848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc38a7b5-3a13-45ca-8e16-dd28104640ed None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:34.944662 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99db1d9-9f76-43d2-b437-95776e72f513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:34.945813 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99db1d9-9f76-43d2-b437-95776e72f513 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:34.946375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99db1d9-9f76-43d2-b437-95776e72f513 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:34.947008 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99db1d9-9f76-43d2-b437-95776e72f513 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:34.948507 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99db1d9-9f76-43d2-b437-95776e72f513 None admin] RBAC: Adding query filter params (group.id=090819f8b1a8447ca66bc3974d538dfb, role.id=33c5141e078844bbb60127f2b26540d7, scope.project.id=459ff8f521284cf8b3ec858bfb1ba630) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:34.949037 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99db1d9-9f76-43d2-b437-95776e72f513 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:34.950989 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99db1d9-9f76-43d2-b437-95776e72f513 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:34.958978 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 211/430] 10.222.0.115 () {56 vars in 1381 bytes} [Tue Aug 2 10:35:34 2022] GET /identity/v3/role_assignments?group.id=090819f8b1a8447ca66bc3974d538dfb&scope.project.id=459ff8f521284cf8b3ec858bfb1ba630&role.id=33c5141e078844bbb60127f2b26540d7 => generated 613 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:34.960227 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99db1d9-9f76-43d2-b437-95776e72f513 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:34.974524 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95621c-bb5a-4e55-89b9-859d02d42dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:34.974909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95621c-bb5a-4e55-89b9-859d02d42dfe None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:34.975118 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95621c-bb5a-4e55-89b9-859d02d42dfe None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:34.975396 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d95621c-bb5a-4e55-89b9-859d02d42dfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:34.979735 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:35.052043 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d95621c-bb5a-4e55-89b9-859d02d42dfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:35.053622 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d95621c-bb5a-4e55-89b9-859d02d42dfe None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:35.085355 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99db1d9-9f76-43d2-b437-95776e72f513 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:35.088126 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:35.128665 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377bff0-547b-41a2-994b-cae23eb59fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:35.130416 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377bff0-547b-41a2-994b-cae23eb59fb4 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:35.131143 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377bff0-547b-41a2-994b-cae23eb59fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:35.131951 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377bff0-547b-41a2-994b-cae23eb59fb4 None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:35.133560 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2377bff0-547b-41a2-994b-cae23eb59fb4 None admin] RBAC: Adding query filter params (name=member) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:35.134347 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2377bff0-547b-41a2-994b-cae23eb59fb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:35.137073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2377bff0-547b-41a2-994b-cae23eb59fb4 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:35.149332 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 212/431] 10.222.0.115 () {56 vars in 1106 bytes} [Tue Aug 2 10:35:34 2022] GET /identity/v3/roles?name=member => generated 343 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:35.162116 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377bff0-547b-41a2-994b-cae23eb59fb4 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:35.168775 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 220/432] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:35:34 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:35:35.174749 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95621c-bb5a-4e55-89b9-859d02d42dfe None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:35.262294 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:35.294333 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377bff0-547b-41a2-994b-cae23eb59fb4 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:35.344381 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac53c2d-f95e-486f-8926-135016b59a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:35.345392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac53c2d-f95e-486f-8926-135016b59a93 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:35.345872 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac53c2d-f95e-486f-8926-135016b59a93 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:35.346401 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac53c2d-f95e-486f-8926-135016b59a93 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:35.346977 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d95621c-bb5a-4e55-89b9-859d02d42dfe None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:35.379118 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:35.384242 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b7053-e9c5-4a9a-a340-0eb654777e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:35.384894 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b7053-e9c5-4a9a-a340-0eb654777e49 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:35.385175 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b7053-e9c5-4a9a-a340-0eb654777e49 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:35.385453 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3b7053-e9c5-4a9a-a340-0eb654777e49 None admin] PATH_INFO: `/v3/projects/459ff8f521284cf8b3ec858bfb1ba630` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:35.386726 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b7053-e9c5-4a9a-a340-0eb654777e49 None admin] RBAC: Authorizing `identity:get_project(project_id=459ff8f521284cf8b3ec858bfb1ba630)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:35.388356 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3b7053-e9c5-4a9a-a340-0eb654777e49 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:35.390348 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 221/433] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:35:35 2022] GET /identity/v3/projects/459ff8f521284cf8b3ec858bfb1ba630 => generated 299 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:35.396222 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b7053-e9c5-4a9a-a340-0eb654777e49 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:35.432802 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac53c2d-f95e-486f-8926-135016b59a93 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:35.434608 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac53c2d-f95e-486f-8926-135016b59a93 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:35.512660 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3b7053-e9c5-4a9a-a340-0eb654777e49 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:35.538547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:35.553432 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec426fe-7b87-48fe-89e8-7ba4e26221ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:35.554442 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec426fe-7b87-48fe-89e8-7ba4e26221ee None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:35.554906 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec426fe-7b87-48fe-89e8-7ba4e26221ee None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:35.555397 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec426fe-7b87-48fe-89e8-7ba4e26221ee None admin] PATH_INFO: `/v3/groups/090819f8b1a8447ca66bc3974d538dfb` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:35.557285 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec426fe-7b87-48fe-89e8-7ba4e26221ee None admin] RBAC: Authorizing `identity:get_group(group_id=090819f8b1a8447ca66bc3974d538dfb)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:35.558982 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec426fe-7b87-48fe-89e8-7ba4e26221ee None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:35.561112 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 222/434] 10.222.0.115 () {56 vars in 1185 bytes} [Tue Aug 2 10:35:35 2022] GET /identity/v3/groups/090819f8b1a8447ca66bc3974d538dfb => generated 227 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:35:35.567384 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec426fe-7b87-48fe-89e8-7ba4e26221ee None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:35.571297 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 213/435] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:35:35 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 10:35:35.573755 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac53c2d-f95e-486f-8926-135016b59a93 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:35.643560 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:35.682611 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec426fe-7b87-48fe-89e8-7ba4e26221ee None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:35.700811 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac53c2d-f95e-486f-8926-135016b59a93 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:35.738379 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c947be-2bd4-4397-b50e-4b2b77c4f286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:35.739022 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c947be-2bd4-4397-b50e-4b2b77c4f286 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:35.739333 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c947be-2bd4-4397-b50e-4b2b77c4f286 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:35.739608 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c947be-2bd4-4397-b50e-4b2b77c4f286 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:35.740413 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c947be-2bd4-4397-b50e-4b2b77c4f286 None admin] RBAC: Adding query filter params (group.id=090819f8b1a8447ca66bc3974d538dfb, role.id=a4213d713b6f49d78d7bbb9484c9312f, scope.project.id=459ff8f521284cf8b3ec858bfb1ba630) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:35.740743 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c947be-2bd4-4397-b50e-4b2b77c4f286 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:35.742186 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c947be-2bd4-4397-b50e-4b2b77c4f286 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:35.749404 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 223/436] 10.222.0.115 () {56 vars in 1381 bytes} [Tue Aug 2 10:35:35 2022] GET /identity/v3/role_assignments?group.id=090819f8b1a8447ca66bc3974d538dfb&scope.project.id=459ff8f521284cf8b3ec858bfb1ba630&role.id=a4213d713b6f49d78d7bbb9484c9312f => generated 613 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:35.750771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4366063-c0c1-46bc-8ee3-1415bf298687 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:35.751079 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4366063-c0c1-46bc-8ee3-1415bf298687 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:35.751384 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4366063-c0c1-46bc-8ee3-1415bf298687 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:35.752911 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 224/437] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:35 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:35.759000 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfaf44c-a748-4403-bb31-cc8be93755a8 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:35.759339 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfaf44c-a748-4403-bb31-cc8be93755a8 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:35.759614 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfaf44c-a748-4403-bb31-cc8be93755a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:35.760971 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca7323-3f5a-4b33-ba47-c5f5e26ecabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:35.761966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca7323-3f5a-4b33-ba47-c5f5e26ecabb None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:35.762598 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca7323-3f5a-4b33-ba47-c5f5e26ecabb None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:35.763157 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca7323-3f5a-4b33-ba47-c5f5e26ecabb None admin] PATH_INFO: `/v3/projects/21b65aad6f794b3588a867ef02e946ac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:35.764734 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ca7323-3f5a-4b33-ba47-c5f5e26ecabb None admin] RBAC: Authorizing `identity:get_project(project_id=21b65aad6f794b3588a867ef02e946ac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:35.767702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ca7323-3f5a-4b33-ba47-c5f5e26ecabb None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:35.773920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 214/438] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:35:35 2022] GET /identity/v3/projects/21b65aad6f794b3588a867ef02e946ac => generated 303 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:35:35.776801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca7323-3f5a-4b33-ba47-c5f5e26ecabb None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:35.780329 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-ccfaf44c-a748-4403-bb31-cc8be93755a8 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:35.782432 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-ccfaf44c-a748-4403-bb31-cc8be93755a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:35.844300 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:35.888228 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 225/439] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:35 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:35.923098 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfaf44c-a748-4403-bb31-cc8be93755a8 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:35.959306 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca7323-3f5a-4b33-ba47-c5f5e26ecabb None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:36.000109 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:36.003884 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ced9e-7a36-48f2-b7a0-d1f1634a7833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:36.004769 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ced9e-7a36-48f2-b7a0-d1f1634a7833 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:36.005125 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ced9e-7a36-48f2-b7a0-d1f1634a7833 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:36.013235 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75ced9e-7a36-48f2-b7a0-d1f1634a7833 None admin] PATH_INFO: `/v3/groups/090819f8b1a8447ca66bc3974d538dfb` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:36.013235 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ced9e-7a36-48f2-b7a0-d1f1634a7833 None admin] RBAC: Authorizing `identity:get_group(group_id=090819f8b1a8447ca66bc3974d538dfb)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:36.013235 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75ced9e-7a36-48f2-b7a0-d1f1634a7833 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:36.013235 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 215/440] 10.222.0.115 () {56 vars in 1185 bytes} [Tue Aug 2 10:35:35 2022] GET /identity/v3/groups/090819f8b1a8447ca66bc3974d538dfb => generated 227 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:36.019787 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ced9e-7a36-48f2-b7a0-d1f1634a7833 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:36.059977 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfaf44c-a748-4403-bb31-cc8be93755a8 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:36.111954 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e9969-a6a7-43e1-9dd9-892adabf53f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:36.112607 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e9969-a6a7-43e1-9dd9-892adabf53f6 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:36.112893 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e9969-a6a7-43e1-9dd9-892adabf53f6 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:36.113176 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0e9969-a6a7-43e1-9dd9-892adabf53f6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:36.134211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0e9969-a6a7-43e1-9dd9-892adabf53f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:36.142994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-da0e9969-a6a7-43e1-9dd9-892adabf53f6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:35:36.142994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:36.142994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:36.142994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:36.142994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:36.142994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:36.142994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:36.142994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:36.142994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:36.142994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:36.142994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:36.142994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:36.142994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:36.142994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:36.142994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:36.142994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:36.144079 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:36.144079 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:36.144079 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:36.144079 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:36.144079 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:36.144079 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:36.144079 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:36.144079 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:36.144079 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:36.144079 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:36.144079 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:36.144079 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:36.144079 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:36.144079 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:36.144079 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:36.144079 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:36.144917 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:36.144917 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:36.144917 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:36.144917 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:36.144917 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:36.144917 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:36.144917 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:36.144917 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:36.144917 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:36.144917 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:36.144917 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:35:36.144917 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:36.144917 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 226/441] 10.222.0.115 () {56 vars in 1119 bytes} [Tue Aug 2 10:35:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:36.150630 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e9969-a6a7-43e1-9dd9-892adabf53f6 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:36.154565 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75ced9e-7a36-48f2-b7a0-d1f1634a7833 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:36.168512 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:36.207674 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c35cfe-d652-49f5-902b-60896cbd2bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:36.213396 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c35cfe-d652-49f5-902b-60896cbd2bbe None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:36.213813 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c35cfe-d652-49f5-902b-60896cbd2bbe None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:36.214200 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c35cfe-d652-49f5-902b-60896cbd2bbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:36.215224 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c35cfe-d652-49f5-902b-60896cbd2bbe None admin] RBAC: Adding query filter params (group.id=090819f8b1a8447ca66bc3974d538dfb, role.id=33c5141e078844bbb60127f2b26540d7, scope.project.id=21b65aad6f794b3588a867ef02e946ac) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:36.215648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c35cfe-d652-49f5-902b-60896cbd2bbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:36.219457 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c35cfe-d652-49f5-902b-60896cbd2bbe None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:36.231587 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 216/442] 10.222.0.115 () {56 vars in 1381 bytes} [Tue Aug 2 10:35:36 2022] GET /identity/v3/role_assignments?group.id=090819f8b1a8447ca66bc3974d538dfb&scope.project.id=21b65aad6f794b3588a867ef02e946ac&role.id=33c5141e078844bbb60127f2b26540d7 => generated 613 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:36.302294 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0e9969-a6a7-43e1-9dd9-892adabf53f6 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:36.332724 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:36.347498 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb36315-d89c-4370-9ee6-dd3b73b81c63 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:36.347713 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6b1ef-3ad5-4021-bb54-8492691686aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:36.347810 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb36315-d89c-4370-9ee6-dd3b73b81c63 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:36.348039 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb36315-d89c-4370-9ee6-dd3b73b81c63 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:36.348132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6b1ef-3ad5-4021-bb54-8492691686aa None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:36.348401 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6b1ef-3ad5-4021-bb54-8492691686aa None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:36.348683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd6b1ef-3ad5-4021-bb54-8492691686aa None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:36.349577 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 217/443] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:35:36.349691 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6b1ef-3ad5-4021-bb54-8492691686aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:36.349765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6b1ef-3ad5-4021-bb54-8492691686aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:36.352794 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd6b1ef-3ad5-4021-bb54-8492691686aa None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:36.355366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f972fa-de54-4d26-b4ba-51427f3ad4ef None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:36.355649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f972fa-de54-4d26-b4ba-51427f3ad4ef None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:36.355923 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f972fa-de54-4d26-b4ba-51427f3ad4ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:36.365260 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 227/444] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:35:36 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:35:36.373370 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6b1ef-3ad5-4021-bb54-8492691686aa None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:36.384974 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-f2f972fa-de54-4d26-b4ba-51427f3ad4ef None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:36.386406 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f2f972fa-de54-4d26-b4ba-51427f3ad4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:36.482575 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 218/445] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:36 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:35:36.502036 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd6b1ef-3ad5-4021-bb54-8492691686aa None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:36.508384 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f972fa-de54-4d26-b4ba-51427f3ad4ef None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:36.540519 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc8b5f-a419-47e5-bdba-438708219879 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:36.541198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc8b5f-a419-47e5-bdba-438708219879 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:36.541478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc8b5f-a419-47e5-bdba-438708219879 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:36.541746 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bc8b5f-a419-47e5-bdba-438708219879 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:36.604441 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:36.624399 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc8b5f-a419-47e5-bdba-438708219879 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:36.627212 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bc8b5f-a419-47e5-bdba-438708219879 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:36.642775 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f972fa-de54-4d26-b4ba-51427f3ad4ef None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:36.645163 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:36.677882 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0948de-de73-433a-aace-aee398ab54d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:36.678533 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0948de-de73-433a-aace-aee398ab54d7 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:36.679112 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0948de-de73-433a-aace-aee398ab54d7 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:36.679617 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0948de-de73-433a-aace-aee398ab54d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:36.691820 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0948de-de73-433a-aace-aee398ab54d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:36.696870 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-3b0948de-de73-433a-aace-aee398ab54d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:35:36.696870 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:36.696870 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:36.696870 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:36.696870 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:36.696870 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:36.696870 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:36.696870 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:36.696870 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:36.696870 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:36.696870 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:36.696870 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:36.696870 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:36.696870 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:36.696870 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:36.696870 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:36.697958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:36.697958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:36.697958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:36.697958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:36.697958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:36.697958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:36.697958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:36.697958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:36.697958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:36.697958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:36.697958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:36.697958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:36.697958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:36.697958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:36.697958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:36.697958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:36.698763 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:36.698763 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:36.698763 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:36.698763 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:36.698763 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:36.698763 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:36.698763 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:36.698763 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:36.698763 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:36.698763 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:36.698763 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:35:36.698763 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:36.698763 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 219/446] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:35:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:35:36.704403 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0948de-de73-433a-aace-aee398ab54d7 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:36.735433 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 228/447] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:35:36 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 10:35:36.740751 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc8b5f-a419-47e5-bdba-438708219879 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:36.814801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0948de-de73-433a-aace-aee398ab54d7 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:36.827176 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:36.843417 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bc8b5f-a419-47e5-bdba-438708219879 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:36.850182 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b76641-1b3e-44e7-b00f-8a31800e1acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:36.850784 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b76641-1b3e-44e7-b00f-8a31800e1acb None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:36.851084 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b76641-1b3e-44e7-b00f-8a31800e1acb None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:36.851353 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b76641-1b3e-44e7-b00f-8a31800e1acb None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:36.852072 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b76641-1b3e-44e7-b00f-8a31800e1acb None admin] RBAC: Adding query filter params (name=admin) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:36.852369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b76641-1b3e-44e7-b00f-8a31800e1acb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:36.853754 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b76641-1b3e-44e7-b00f-8a31800e1acb None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:36.863942 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 220/448] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:35:36 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:36.869190 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b76641-1b3e-44e7-b00f-8a31800e1acb None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:36.881375 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a850e-7197-45dc-b9d0-41f7667fa113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:36.882026 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a850e-7197-45dc-b9d0-41f7667fa113 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:36.882303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a850e-7197-45dc-b9d0-41f7667fa113 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:36.882581 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a850e-7197-45dc-b9d0-41f7667fa113 None admin] PATH_INFO: `/v3/projects/21b65aad6f794b3588a867ef02e946ac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:36.883776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8a850e-7197-45dc-b9d0-41f7667fa113 None admin] RBAC: Authorizing `identity:get_project(project_id=21b65aad6f794b3588a867ef02e946ac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:36.885261 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8a850e-7197-45dc-b9d0-41f7667fa113 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:36.887083 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 229/449] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:35:36 2022] GET /identity/v3/projects/21b65aad6f794b3588a867ef02e946ac => generated 303 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:35:36.892688 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a850e-7197-45dc-b9d0-41f7667fa113 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:36.899808 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:36.980457 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b76641-1b3e-44e7-b00f-8a31800e1acb None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:37.004033 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a850e-7197-45dc-b9d0-41f7667fa113 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:37.016735 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c8e4c0-ae6e-49dd-b768-4d13301c3405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:37.017411 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8e4c0-ae6e-49dd-b768-4d13301c3405 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:37.017688 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8e4c0-ae6e-49dd-b768-4d13301c3405 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:37.017962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c8e4c0-ae6e-49dd-b768-4d13301c3405 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:37.044398 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c788e-9310-49bc-993d-e1bbe9c3dcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:37.045036 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c788e-9310-49bc-993d-e1bbe9c3dcec None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:37.045314 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c788e-9310-49bc-993d-e1bbe9c3dcec None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:37.045587 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07c788e-9310-49bc-993d-e1bbe9c3dcec None admin] PATH_INFO: `/v3/groups/090819f8b1a8447ca66bc3974d538dfb` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:37.046800 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c788e-9310-49bc-993d-e1bbe9c3dcec None admin] RBAC: Authorizing `identity:get_group(group_id=090819f8b1a8447ca66bc3974d538dfb)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:37.048833 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07c788e-9310-49bc-993d-e1bbe9c3dcec None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:37.050462 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 230/450] 10.222.0.115 () {56 vars in 1185 bytes} [Tue Aug 2 10:35:36 2022] GET /identity/v3/groups/090819f8b1a8447ca66bc3974d538dfb => generated 227 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:35:37.056372 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c788e-9310-49bc-993d-e1bbe9c3dcec None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:37.079145 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:37.092631 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c8e4c0-ae6e-49dd-b768-4d13301c3405 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:37.094582 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c8e4c0-ae6e-49dd-b768-4d13301c3405 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:37.121672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:37.178742 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07c788e-9310-49bc-993d-e1bbe9c3dcec None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:37.197561 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 221/451] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:35:36 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 02 10:35:37.203949 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c8e4c0-ae6e-49dd-b768-4d13301c3405 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:37.221553 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59561ba-06f7-4d44-a73f-f426f4ae3b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:37.222421 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59561ba-06f7-4d44-a73f-f426f4ae3b57 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:37.222698 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59561ba-06f7-4d44-a73f-f426f4ae3b57 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:37.222967 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59561ba-06f7-4d44-a73f-f426f4ae3b57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:37.223815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59561ba-06f7-4d44-a73f-f426f4ae3b57 None admin] RBAC: Adding query filter params (group.id=090819f8b1a8447ca66bc3974d538dfb, role.id=a4213d713b6f49d78d7bbb9484c9312f, scope.project.id=21b65aad6f794b3588a867ef02e946ac) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:37.224524 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59561ba-06f7-4d44-a73f-f426f4ae3b57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:37.225514 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59561ba-06f7-4d44-a73f-f426f4ae3b57 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:37.231516 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 231/452] 10.222.0.115 () {56 vars in 1381 bytes} [Tue Aug 2 10:35:37 2022] GET /identity/v3/role_assignments?group.id=090819f8b1a8447ca66bc3974d538dfb&scope.project.id=21b65aad6f794b3588a867ef02e946ac&role.id=a4213d713b6f49d78d7bbb9484c9312f => generated 613 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:35:37.314873 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c8e4c0-ae6e-49dd-b768-4d13301c3405 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:37.344466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:37.354727 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0cc29-cf3c-405a-98d9-b3c22d917143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:37.355421 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0cc29-cf3c-405a-98d9-b3c22d917143 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:37.355704 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0cc29-cf3c-405a-98d9-b3c22d917143 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:37.355980 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b0cc29-cf3c-405a-98d9-b3c22d917143 None admin] PATH_INFO: `/v3/projects/b86f954a6cac457bad0b226e93315c59` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:37.357201 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b0cc29-cf3c-405a-98d9-b3c22d917143 None admin] RBAC: Authorizing `identity:get_project(project_id=b86f954a6cac457bad0b226e93315c59)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:37.358680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b0cc29-cf3c-405a-98d9-b3c22d917143 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:37.361666 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 222/453] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:35:37 2022] GET /identity/v3/projects/b86f954a6cac457bad0b226e93315c59 => generated 345 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:37.366715 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59561ba-06f7-4d44-a73f-f426f4ae3b57 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:37.468217 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:37.485337 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59561ba-06f7-4d44-a73f-f426f4ae3b57 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:37.523562 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fd1ca5-74c7-4041-af2e-e04afe534ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:37.524219 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd1ca5-74c7-4041-af2e-e04afe534ef1 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:37.524506 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd1ca5-74c7-4041-af2e-e04afe534ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:37.524789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fd1ca5-74c7-4041-af2e-e04afe534ef1 None admin] PATH_INFO: `/v3/groups/fc14f4d6ca474719a326c7ff8596c2c9` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:37.525958 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd1ca5-74c7-4041-af2e-e04afe534ef1 None admin] RBAC: Authorizing `identity:get_group(group_id=fc14f4d6ca474719a326c7ff8596c2c9)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:37.527547 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18fd1ca5-74c7-4041-af2e-e04afe534ef1 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:37.529162 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 232/454] 10.222.0.115 () {56 vars in 1185 bytes} [Tue Aug 2 10:35:37 2022] GET /identity/v3/groups/fc14f4d6ca474719a326c7ff8596c2c9 => generated 230 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:37.535409 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0cc29-cf3c-405a-98d9-b3c22d917143 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:37.645547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b0cc29-cf3c-405a-98d9-b3c22d917143 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:37.681913 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc43cd2-7415-4807-a2ac-84149909576f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:37.682924 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc43cd2-7415-4807-a2ac-84149909576f None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:37.683464 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc43cd2-7415-4807-a2ac-84149909576f None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:37.683971 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc43cd2-7415-4807-a2ac-84149909576f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:37.685438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc43cd2-7415-4807-a2ac-84149909576f None admin] RBAC: Adding query filter params (group.id=fc14f4d6ca474719a326c7ff8596c2c9, role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac, scope.project.id=b86f954a6cac457bad0b226e93315c59) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:37.685990 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc43cd2-7415-4807-a2ac-84149909576f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:37.688292 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc43cd2-7415-4807-a2ac-84149909576f None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:37.694481 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 223/455] 10.222.0.115 () {56 vars in 1381 bytes} [Tue Aug 2 10:35:37 2022] GET /identity/v3/role_assignments?group.id=fc14f4d6ca474719a326c7ff8596c2c9&scope.project.id=b86f954a6cac457bad0b226e93315c59&role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 613 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:40.184523 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afb029-8ebc-4fdb-97f1-c62161f5e3aa None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:40.185304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afb029-8ebc-4fdb-97f1-c62161f5e3aa None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:40.185304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26afb029-8ebc-4fdb-97f1-c62161f5e3aa None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:40.186672 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 233/456] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:40.192785 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1903c7-bfb3-4a05-8bf1-efdfadaf0fb9 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:40.193450 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1903c7-bfb3-4a05-8bf1-efdfadaf0fb9 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:40.193450 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1903c7-bfb3-4a05-8bf1-efdfadaf0fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:40.219789 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-fb1903c7-bfb3-4a05-8bf1-efdfadaf0fb9 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:40.221754 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-fb1903c7-bfb3-4a05-8bf1-efdfadaf0fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:40.339287 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:40.363934 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 224/457] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:40 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:40.405749 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afb029-8ebc-4fdb-97f1-c62161f5e3aa None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:40.695943 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26afb029-8ebc-4fdb-97f1-c62161f5e3aa None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:40.791877 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368bd23-ddb3-48e3-bef2-5715eff17f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:40.792559 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368bd23-ddb3-48e3-bef2-5715eff17f24 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:40.792845 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368bd23-ddb3-48e3-bef2-5715eff17f24 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:40.793119 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7368bd23-ddb3-48e3-bef2-5715eff17f24 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:40.828795 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7368bd23-ddb3-48e3-bef2-5715eff17f24 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:40.830306 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7368bd23-ddb3-48e3-bef2-5715eff17f24 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:40.864585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-7368bd23-ddb3-48e3-bef2-5715eff17f24 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:40.864585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:40.864585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:40.864585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:40.864585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:40.864585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:40.864585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:40.864585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:40.864585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:40.864585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:40.864585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:40.864585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:40.864585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:35:40.864585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:35:40.864585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:35:40.865596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:35:40.865596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:40.865596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:40.865596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:40.865596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:40.865596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:40.865596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:40.865596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:40.865596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:40.865596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:40.865596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:40.865596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:40.865596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:40.865596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:40.865596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:40.865596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:40.865596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:40.866503 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:35:40.866503 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:35:40.866503 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:40.866503 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:40.866503 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 234/458] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 461 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:40.875139 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1903c7-bfb3-4a05-8bf1-efdfadaf0fb9 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:41.044251 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1903c7-bfb3-4a05-8bf1-efdfadaf0fb9 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:41.111801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab23692-4191-45bd-b527-d9b164e19e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:41.112816 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab23692-4191-45bd-b527-d9b164e19e61 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:41.113398 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab23692-4191-45bd-b527-d9b164e19e61 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:41.113878 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab23692-4191-45bd-b527-d9b164e19e61 None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:41.114789 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab23692-4191-45bd-b527-d9b164e19e61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:41.115320 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab23692-4191-45bd-b527-d9b164e19e61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:41.116902 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab23692-4191-45bd-b527-d9b164e19e61 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:41.134508 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 225/459] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:35:40 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:41.143547 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368bd23-ddb3-48e3-bef2-5715eff17f24 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:41.148787 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:41.255916 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960eb29-e57b-4a05-9e0c-9d2ed741342c None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:41.256445 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960eb29-e57b-4a05-9e0c-9d2ed741342c None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:41.256445 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0960eb29-e57b-4a05-9e0c-9d2ed741342c None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:41.258104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 226/460] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:35:41.263445 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57395ed-34e2-45e4-b65f-37c83890e07f None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:41.263729 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57395ed-34e2-45e4-b65f-37c83890e07f None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:41.264008 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57395ed-34e2-45e4-b65f-37c83890e07f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:41.283058 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-d57395ed-34e2-45e4-b65f-37c83890e07f None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:41.284274 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-d57395ed-34e2-45e4-b65f-37c83890e07f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:41.291778 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7368bd23-ddb3-48e3-bef2-5715eff17f24 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:41.334337 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36172f39-bf85-4a7f-a24a-6c9d47ecc7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:41.334884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36172f39-bf85-4a7f-a24a-6c9d47ecc7b5 None admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:41.335218 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36172f39-bf85-4a7f-a24a-6c9d47ecc7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:41.335340 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:41.335500 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36172f39-bf85-4a7f-a24a-6c9d47ecc7b5 None admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:41.336521 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36172f39-bf85-4a7f-a24a-6c9d47ecc7b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:41.338287 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36172f39-bf85-4a7f-a24a-6c9d47ecc7b5 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:41.361949 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 227/461] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:41 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:35:41.363259 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0f981-938f-4a11-8e7c-363d3db28433 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:41.363538 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0f981-938f-4a11-8e7c-363d3db28433 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:41.363809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf0f981-938f-4a11-8e7c-363d3db28433 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:41.365354 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 228/462] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:41.371796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53db1db-b70a-4291-ab4d-608d4e9d9159 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:41.372076 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53db1db-b70a-4291-ab4d-608d4e9d9159 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:41.372350 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53db1db-b70a-4291-ab4d-608d4e9d9159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:41.392458 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-a53db1db-b70a-4291-ab4d-608d4e9d9159 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:41.393646 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-a53db1db-b70a-4291-ab4d-608d4e9d9159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:41.419293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 235/463] 10.222.0.115 () {60 vars in 1137 bytes} [Tue Aug 2 10:35:41 2022] POST /identity/v3/users => generated 247 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:35:41.422332 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36172f39-bf85-4a7f-a24a-6c9d47ecc7b5 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:41.519621 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 229/464] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:41 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:35:41.527868 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5b8e9-78cc-454f-8efc-4e214604f91c None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:41.528193 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5b8e9-78cc-454f-8efc-4e214604f91c None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:41.528419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d5b8e9-78cc-454f-8efc-4e214604f91c None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:41.532013 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 230/465] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:41 2022] GET /identity => generated 271 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:41.537234 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697cfc9d-c15c-4c66-b27a-c8379c3b8855 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:41.537513 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697cfc9d-c15c-4c66-b27a-c8379c3b8855 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:41.537789 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697cfc9d-c15c-4c66-b27a-c8379c3b8855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:41.543630 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:41.567210 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-697cfc9d-c15c-4c66-b27a-c8379c3b8855 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:41.568966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-697cfc9d-c15c-4c66-b27a-c8379c3b8855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:41.600776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36172f39-bf85-4a7f-a24a-6c9d47ecc7b5 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:41.652529 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be596bb-eb55-44b2-82e1-e7118a7c3a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:41.653489 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be596bb-eb55-44b2-82e1-e7118a7c3a10 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:41.653772 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be596bb-eb55-44b2-82e1-e7118a7c3a10 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:41.654045 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be596bb-eb55-44b2-82e1-e7118a7c3a10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:41.663393 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 231/466] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:41 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:41.664294 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697cfc9d-c15c-4c66-b27a-c8379c3b8855 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:41.667021 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be596bb-eb55-44b2-82e1-e7118a7c3a10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:41.668545 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be596bb-eb55-44b2-82e1-e7118a7c3a10 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:41.688163 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-7be596bb-eb55-44b2-82e1-e7118a7c3a10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:41.688163 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:41.688163 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:41.688163 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:41.688163 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:41.688163 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:41.688163 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:41.688163 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:41.688163 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:41.688163 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:41.688163 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:41.688163 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:41.688163 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:35:41.688163 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:35:41.688163 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:35:41.689106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:35:41.689106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:41.689106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:41.689106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:41.689106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:41.689106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:41.689106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:41.689106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:41.689106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:41.689106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:41.689106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:41.689106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:41.689106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:41.689106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:41.689106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:41.689106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:41.689106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:41.690032 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:35:41.690032 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:35:41.690032 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:41.690032 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:41.693304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 236/467] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:41.696230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be596bb-eb55-44b2-82e1-e7118a7c3a10 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:41.724812 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:41.809561 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697cfc9d-c15c-4c66-b27a-c8379c3b8855 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:41.810945 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be596bb-eb55-44b2-82e1-e7118a7c3a10 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:41.825865 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:41.851685 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43127f-9524-452a-b908-4a922e53556c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:41.852736 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43127f-9524-452a-b908-4a922e53556c None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:41.853122 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43127f-9524-452a-b908-4a922e53556c None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:41.853440 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c43127f-9524-452a-b908-4a922e53556c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:41.857679 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d8c7e1-e29d-4121-9016-42700d4564c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:41.858329 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8c7e1-e29d-4121-9016-42700d4564c7 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:41.858658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8c7e1-e29d-4121-9016-42700d4564c7 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:41.859599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8c7e1-e29d-4121-9016-42700d4564c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:41.867554 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c43127f-9524-452a-b908-4a922e53556c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:41.869105 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c43127f-9524-452a-b908-4a922e53556c None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:41.873013 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d8c7e1-e29d-4121-9016-42700d4564c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:41.874487 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d8c7e1-e29d-4121-9016-42700d4564c7 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:41.886568 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-5c43127f-9524-452a-b908-4a922e53556c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:41.886568 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:41.886568 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:41.886568 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:41.886568 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:41.886568 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:41.886568 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:41.886568 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:41.886568 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:41.886568 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:41.886568 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:41.886568 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:41.886568 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:35:41.886568 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:35:41.886568 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:35:41.887561 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:35:41.887561 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:41.887561 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:41.887561 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:41.887561 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:41.887561 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:41.887561 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:41.887561 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:41.887561 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:41.887561 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:41.887561 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:41.887561 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:41.887561 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:41.887561 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:41.887561 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:41.887561 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:41.887561 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:41.888441 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:35:41.888441 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:35:41.888441 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:41.888441 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:41.888441 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 237/468] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:41.890193 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43127f-9524-452a-b908-4a922e53556c None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:41.896152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-73d8c7e1-e29d-4121-9016-42700d4564c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:41.896152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:41.896152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:41.896152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:41.896152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:41.896152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:41.896152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:41.896152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:41.896152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:41.896152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:41.896152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:41.896152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:41.896152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:35:41.896152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:35:41.896152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:35:41.897066 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:35:41.897066 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:41.897066 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:41.897066 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:41.897066 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:41.897066 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:41.897066 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:41.897066 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:41.897066 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:41.897066 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:41.897066 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:41.897066 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:41.897066 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:41.897066 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:41.897066 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:41.897066 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:41.897066 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:41.897916 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:35:41.897916 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:35:41.897916 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:41.897916 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:41.897916 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 232/469] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:41.898904 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d8c7e1-e29d-4121-9016-42700d4564c7 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:42.014970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:42.025030 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d8c7e1-e29d-4121-9016-42700d4564c7 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:42.036030 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c43127f-9524-452a-b908-4a922e53556c None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:42.067464 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a58a62-c884-4986-90d1-7092caa24b9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:42.068322 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a58a62-c884-4986-90d1-7092caa24b9f None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:42.068772 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a58a62-c884-4986-90d1-7092caa24b9f None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:42.069045 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a58a62-c884-4986-90d1-7092caa24b9f None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:42.069743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a58a62-c884-4986-90d1-7092caa24b9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:42.070032 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a58a62-c884-4986-90d1-7092caa24b9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:42.071445 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a58a62-c884-4986-90d1-7092caa24b9f None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:42.084479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab9f35-d12c-476b-a0a5-03d7d930bfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:42.084923 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 233/470] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:35:41 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:42.085689 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab9f35-d12c-476b-a0a5-03d7d930bfc2 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:42.085795 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a58a62-c884-4986-90d1-7092caa24b9f None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:42.086313 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab9f35-d12c-476b-a0a5-03d7d930bfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:42.086803 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ab9f35-d12c-476b-a0a5-03d7d930bfc2 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:42.088507 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ab9f35-d12c-476b-a0a5-03d7d930bfc2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:42.089084 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ab9f35-d12c-476b-a0a5-03d7d930bfc2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:42.091711 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ab9f35-d12c-476b-a0a5-03d7d930bfc2 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:42.112290 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 238/471] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:35:41 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:42.112290 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab9f35-d12c-476b-a0a5-03d7d930bfc2 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:42.147728 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:42.211458 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a58a62-c884-4986-90d1-7092caa24b9f None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:42.256113 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71737db7-0529-410e-90bc-4c0a948e675c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:42.257007 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71737db7-0529-410e-90bc-4c0a948e675c None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:42.257289 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71737db7-0529-410e-90bc-4c0a948e675c None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:42.257566 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71737db7-0529-410e-90bc-4c0a948e675c None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:42.258849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71737db7-0529-410e-90bc-4c0a948e675c None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:42.259317 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71737db7-0529-410e-90bc-4c0a948e675c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:42.261373 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71737db7-0529-410e-90bc-4c0a948e675c None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:42.263952 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ab9f35-d12c-476b-a0a5-03d7d930bfc2 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:42.278403 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 234/472] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:35:42 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:42.279356 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71737db7-0529-410e-90bc-4c0a948e675c None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:42.296036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:42.306848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517d55b-9435-4e9e-8acc-3dca5773a45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:42.317617 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517d55b-9435-4e9e-8acc-3dca5773a45f None admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:42.317617 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517d55b-9435-4e9e-8acc-3dca5773a45f None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:42.317617 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517d55b-9435-4e9e-8acc-3dca5773a45f None admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:42.317617 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517d55b-9435-4e9e-8acc-3dca5773a45f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:42.317617 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9517d55b-9435-4e9e-8acc-3dca5773a45f None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:42.403288 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71737db7-0529-410e-90bc-4c0a948e675c None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:42.414258 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 239/473] 10.222.0.115 () {60 vars in 1137 bytes} [Tue Aug 2 10:35:42 2022] POST /identity/v3/users => generated 249 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:35:42.415165 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517d55b-9435-4e9e-8acc-3dca5773a45f None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:42.467366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd3cae-be65-4281-9b47-36c0ae292228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:42.468011 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd3cae-be65-4281-9b47-36c0ae292228 None admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:42.468854 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd3cae-be65-4281-9b47-36c0ae292228 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:42.470212 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd3cae-be65-4281-9b47-36c0ae292228 None admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:42.470769 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd3cae-be65-4281-9b47-36c0ae292228 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:42.473023 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd3cae-be65-4281-9b47-36c0ae292228 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:42.551696 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:42.576930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517d55b-9435-4e9e-8acc-3dca5773a45f None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:42.624305 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c809bc4-5464-4def-a469-87b91f8f7c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:42.624988 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c809bc4-5464-4def-a469-87b91f8f7c56 None admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:42.625273 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c809bc4-5464-4def-a469-87b91f8f7c56 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:42.625554 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c809bc4-5464-4def-a469-87b91f8f7c56 None admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:42.626244 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c809bc4-5464-4def-a469-87b91f8f7c56 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:42.627983 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c809bc4-5464-4def-a469-87b91f8f7c56 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:42.704884 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 235/474] 10.222.0.115 () {60 vars in 1137 bytes} [Tue Aug 2 10:35:42 2022] POST /identity/v3/users => generated 250 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:35:42.707487 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea6515-fc09-4ec0-a7fd-55585fbcdcf3 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:42.708350 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea6515-fc09-4ec0-a7fd-55585fbcdcf3 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:42.709095 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ea6515-fc09-4ec0-a7fd-55585fbcdcf3 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:42.715145 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 236/475] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:42 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:42.721022 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c914c-f899-425d-813e-633ba54c20e0 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:42.721597 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c914c-f899-425d-813e-633ba54c20e0 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:42.721868 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c914c-f899-425d-813e-633ba54c20e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:42.734221 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 240/476] 10.222.0.115 () {60 vars in 1137 bytes} [Tue Aug 2 10:35:42 2022] POST /identity/v3/users => generated 249 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:35:42.760621 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-854c914c-f899-425d-813e-633ba54c20e0 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:42.762069 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-854c914c-f899-425d-813e-633ba54c20e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:42.877240 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:42.886263 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 237/477] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:42 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:42.923836 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c809bc4-5464-4def-a469-87b91f8f7c56 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:43.054908 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c809bc4-5464-4def-a469-87b91f8f7c56 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:43.092894 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d31aec-f627-45db-a898-f57e5d1f3cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:43.094048 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d31aec-f627-45db-a898-f57e5d1f3cd5 None admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:43.094318 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d31aec-f627-45db-a898-f57e5d1f3cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:43.094602 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d31aec-f627-45db-a898-f57e5d1f3cd5 None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:43.095374 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d31aec-f627-45db-a898-f57e5d1f3cd5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:43.096770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d31aec-f627-45db-a898-f57e5d1f3cd5 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:43.102106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:43.188228 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.sql.core [None req-a3d31aec-f627-45db-a898-f57e5d1f3cd5 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 02 10:35:43.188228 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 02 10:35:43.188228 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [parameters: {'id': '6d1c1de21a1c47d3b99a6a2c8597ac93', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 02 10:35:43.188228 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=92110) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 02 10:35:43.209940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-a3d31aec-f627-45db-a898-f57e5d1f3cd5 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 02 10:35:43.209940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:43.209940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 10:35:43.209940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 10:35:43.209940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 10:35:43.209940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 10:35:43.209940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 10:35:43.209940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 10:35:43.209940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 10:35:43.209940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application conn.query(q) Aug 02 10:35:43.209940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 10:35:43.209940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 10:35:43.209940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 10:35:43.209940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application result.read() Aug 02 10:35:43.209940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 10:35:43.212626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 10:35:43.212626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 10:35:43.212626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 10:35:43.212626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 10:35:43.212626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 10:35:43.212626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 10:35:43.212626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 10:35:43.212626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 02 10:35:43.212626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:43.212626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 02 10:35:43.212626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:43.212626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:43.212626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 02 10:35:43.212626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 02 10:35:43.212626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 02 10:35:43.212626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return ref.to_dict() Aug 02 10:35:43.212626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 10:35:43.212626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application next(self.gen) Aug 02 10:35:43.215967 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 02 10:35:43.215967 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application yield resource Aug 02 10:35:43.215967 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 10:35:43.215967 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application next(self.gen) Aug 02 10:35:43.215967 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 02 10:35:43.215967 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self.session.rollback() Aug 02 10:35:43.215967 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 02 10:35:43.215967 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self.force_reraise() Aug 02 10:35:43.215967 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 02 10:35:43.215967 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise self.value Aug 02 10:35:43.215967 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 02 10:35:43.215967 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 02 10:35:43.215967 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 02 10:35:43.215967 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application session.commit() Aug 02 10:35:43.215967 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 02 10:35:43.215967 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 02 10:35:43.215967 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 02 10:35:43.223151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._prepare_impl() Aug 02 10:35:43.223151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 02 10:35:43.223151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self.session.flush() Aug 02 10:35:43.223151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 02 10:35:43.223151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._flush(objects) Aug 02 10:35:43.223151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 02 10:35:43.223151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 02 10:35:43.223151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 02 10:35:43.223151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application compat.raise_( Aug 02 10:35:43.223151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 10:35:43.223151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception Aug 02 10:35:43.223151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 02 10:35:43.223151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application flush_context.execute() Aug 02 10:35:43.223151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 02 10:35:43.223151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rec.execute(self) Aug 02 10:35:43.223151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 02 10:35:43.223151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 02 10:35:43.225727 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 02 10:35:43.225727 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 02 10:35:43.225727 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 02 10:35:43.225727 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 02 10:35:43.225727 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 02 10:35:43.225727 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 02 10:35:43.225727 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 02 10:35:43.225727 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 02 10:35:43.225727 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 02 10:35:43.225727 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 02 10:35:43.225727 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 02 10:35:43.225727 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 02 10:35:43.225727 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 02 10:35:43.225727 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 02 10:35:43.225727 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 10:35:43.225727 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception Aug 02 10:35:43.227735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 10:35:43.227735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 10:35:43.227735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 10:35:43.227735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 10:35:43.227735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 10:35:43.227735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 10:35:43.227735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 10:35:43.227735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application conn.query(q) Aug 02 10:35:43.227735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 10:35:43.227735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 10:35:43.227735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 10:35:43.227735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application result.read() Aug 02 10:35:43.227735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 10:35:43.227735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 10:35:43.227735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 10:35:43.227735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 10:35:43.229799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 10:35:43.229799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 10:35:43.229799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 10:35:43.229799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 10:35:43.229799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 02 10:35:43.229799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 02 10:35:43.229799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [parameters: {'id': '6d1c1de21a1c47d3b99a6a2c8597ac93', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 02 10:35:43.229799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 02 10:35:43.229799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:43.229799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:35:43.229799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:43.229799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:43.229799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:43.229799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:43.229799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:43.229799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:43.231927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:43.231927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:43.231927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:43.231927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:43.231927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:43.231927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:43.231927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 02 10:35:43.231927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 02 10:35:43.231927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:43.231927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:43.231927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 02 10:35:43.231927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 02 10:35:43.231927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 02 10:35:43.231927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 02 10:35:43.231927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 02 10:35:43.231927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:43.234783 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 241/478] 10.222.0.115 () {60 vars in 1136 bytes} [Tue Aug 2 10:35:42 2022] POST /identity/v3/roles => generated 144 bytes in 289 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:43.234997 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c914c-f899-425d-813e-633ba54c20e0 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:43.351267 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854c914c-f899-425d-813e-633ba54c20e0 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:43.394839 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b135f1-6d60-47ad-af9d-8ad5b48301af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:43.395521 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b135f1-6d60-47ad-af9d-8ad5b48301af None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:43.395801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b135f1-6d60-47ad-af9d-8ad5b48301af None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:43.396076 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b135f1-6d60-47ad-af9d-8ad5b48301af None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:43.407829 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b135f1-6d60-47ad-af9d-8ad5b48301af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:43.412908 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-e7b135f1-6d60-47ad-af9d-8ad5b48301af None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:35:43.412908 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:43.412908 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:43.412908 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:43.412908 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:43.412908 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:43.412908 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:43.412908 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:43.412908 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:43.412908 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:43.412908 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:43.412908 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:43.412908 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:43.412908 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:43.412908 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:43.412908 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:43.415010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:43.415010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:43.415010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:43.415010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:43.415010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:43.415010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:43.415010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:43.415010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:43.415010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:43.415010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:43.415010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:43.415010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:43.415010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:43.415010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:43.415010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:43.415010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:43.416822 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:43.416822 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:43.416822 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:43.416822 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:43.416822 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:43.416822 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:43.416822 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:43.416822 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:43.416822 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:43.416822 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:43.416822 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 02 10:35:43.416822 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:43.416822 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 238/479] 10.222.0.115 () {56 vars in 1119 bytes} [Tue Aug 2 10:35:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:43.420244 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d31aec-f627-45db-a898-f57e5d1f3cd5 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:43.603754 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d31aec-f627-45db-a898-f57e5d1f3cd5 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:43.642630 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1506a316-f4a8-4488-a316-9bfc524ea389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:43.643324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506a316-f4a8-4488-a316-9bfc524ea389 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:43.643605 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506a316-f4a8-4488-a316-9bfc524ea389 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:43.643878 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1506a316-f4a8-4488-a316-9bfc524ea389 None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:43.644607 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1506a316-f4a8-4488-a316-9bfc524ea389 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:43.644917 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1506a316-f4a8-4488-a316-9bfc524ea389 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:43.646388 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1506a316-f4a8-4488-a316-9bfc524ea389 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:43.655230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 242/480] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:35:43 2022] GET /identity/v3/roles?name=anotherrole => generated 336 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:43.994745 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac780e-4767-42d6-ad53-5e578f1fc0ae None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:43.995102 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac780e-4767-42d6-ad53-5e578f1fc0ae None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:43.995273 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac780e-4767-42d6-ad53-5e578f1fc0ae None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:43.997119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 239/481] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:44.004905 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcad4d-dc0c-4561-83f3-2f46ad96c575 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:44.004905 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcad4d-dc0c-4561-83f3-2f46ad96c575 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:44.004905 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fcad4d-dc0c-4561-83f3-2f46ad96c575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:44.025799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-75fcad4d-dc0c-4561-83f3-2f46ad96c575 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:44.026985 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-75fcad4d-dc0c-4561-83f3-2f46ad96c575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:44.098865 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:44.111214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 243/482] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:44 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:44.142576 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac780e-4767-42d6-ad53-5e578f1fc0ae None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:44.235441 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:44.292672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac780e-4767-42d6-ad53-5e578f1fc0ae None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:44.344849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a7ba5b-93b7-47a8-b722-028729b8d90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:44.345526 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7ba5b-93b7-47a8-b722-028729b8d90e None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:44.345800 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7ba5b-93b7-47a8-b722-028729b8d90e None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:44.346069 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a7ba5b-93b7-47a8-b722-028729b8d90e None admin] PATH_INFO: `/v3/roles/service` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:44.365712 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a7ba5b-93b7-47a8-b722-028729b8d90e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:44.370892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-b2a7ba5b-93b7-47a8-b722-028729b8d90e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:35:44.370892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:44.370892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:44.370892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:44.370892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:44.370892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:44.370892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:44.370892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:44.370892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:44.370892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:44.370892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:44.370892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:44.370892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:44.370892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:44.370892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:44.370892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:44.371872 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:44.371872 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:44.371872 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:44.371872 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:44.371872 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:44.371872 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:44.371872 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:44.371872 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:44.371872 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:44.371872 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:44.371872 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:44.371872 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:44.371872 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:44.371872 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:44.371872 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:44.371872 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:44.372672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:44.372672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:44.372672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:44.372672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:44.372672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:44.372672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:44.372672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:44.372672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:44.372672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:44.372672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:44.372672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:35:44.372672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:44.374400 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 240/483] 10.222.0.115 () {56 vars in 1107 bytes} [Tue Aug 2 10:35:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:44.380094 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fcad4d-dc0c-4561-83f3-2f46ad96c575 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:44.537545 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fcad4d-dc0c-4561-83f3-2f46ad96c575 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:44.598208 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ed17a-90da-466e-800d-0184dcd5e6f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:44.598833 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ed17a-90da-466e-800d-0184dcd5e6f0 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:44.599113 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ed17a-90da-466e-800d-0184dcd5e6f0 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:44.599390 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4ed17a-90da-466e-800d-0184dcd5e6f0 None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:44.600119 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ed17a-90da-466e-800d-0184dcd5e6f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:44.600424 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ed17a-90da-466e-800d-0184dcd5e6f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:44.601809 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4ed17a-90da-466e-800d-0184dcd5e6f0 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:44.614851 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 244/484] 10.222.0.115 () {56 vars in 1108 bytes} [Tue Aug 2 10:35:44 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:44.620348 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a7ba5b-93b7-47a8-b722-028729b8d90e None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:44.761541 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a7ba5b-93b7-47a8-b722-028729b8d90e None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:44.787716 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:44.815547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53a3d34-50a9-40f5-98a5-be01ce8d2e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:44.816536 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a3d34-50a9-40f5-98a5-be01ce8d2e56 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:44.817008 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a3d34-50a9-40f5-98a5-be01ce8d2e56 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:44.817462 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53a3d34-50a9-40f5-98a5-be01ce8d2e56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:44.832522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a3d34-50a9-40f5-98a5-be01ce8d2e56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:44.834297 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53a3d34-50a9-40f5-98a5-be01ce8d2e56 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:44.854887 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-a53a3d34-50a9-40f5-98a5-be01ce8d2e56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:44.854887 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:44.854887 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:44.854887 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:44.854887 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:44.854887 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:44.854887 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:44.854887 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:44.854887 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:44.854887 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:44.854887 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:44.854887 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:44.854887 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:35:44.854887 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:35:44.854887 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:35:44.860405 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:35:44.860405 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:44.860405 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:44.860405 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:44.860405 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:44.860405 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:44.860405 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:44.860405 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:44.860405 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:44.860405 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:44.860405 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:44.860405 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:44.860405 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:44.860405 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:44.860405 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:44.860405 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:44.860405 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:44.861530 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:35:44.861530 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:35:44.861530 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:44.861530 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:44.863321 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 241/485] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:44.871813 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ed17a-90da-466e-800d-0184dcd5e6f0 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:44.979450 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:45.054009 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4ed17a-90da-466e-800d-0184dcd5e6f0 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:45.106703 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc41138-3678-40d9-a88a-7cea0f291d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:45.107590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc41138-3678-40d9-a88a-7cea0f291d95 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:45.108040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc41138-3678-40d9-a88a-7cea0f291d95 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:45.108487 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc41138-3678-40d9-a88a-7cea0f291d95 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:45.109903 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc41138-3678-40d9-a88a-7cea0f291d95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:45.110458 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc41138-3678-40d9-a88a-7cea0f291d95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:45.112027 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc41138-3678-40d9-a88a-7cea0f291d95 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:45.120664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a5d56-4c58-4411-b9b2-9ee33d6a606a None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:45.121304 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a5d56-4c58-4411-b9b2-9ee33d6a606a None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:45.121885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3a5d56-4c58-4411-b9b2-9ee33d6a606a None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:45.126424 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 242/486] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:45 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:45.132299 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 245/487] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:35:44 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:45.133063 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6a28b-c45f-49a8-aa87-c0625be39862 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:45.133444 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6a28b-c45f-49a8-aa87-c0625be39862 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:45.133815 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d6a28b-c45f-49a8-aa87-c0625be39862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:45.138257 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc41138-3678-40d9-a88a-7cea0f291d95 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:45.173122 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-50d6a28b-c45f-49a8-aa87-c0625be39862 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:45.174442 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-50d6a28b-c45f-49a8-aa87-c0625be39862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:45.269551 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc41138-3678-40d9-a88a-7cea0f291d95 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:45.296143 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:45.301279 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 243/488] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:45 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:45.329559 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef2f67-b7dc-4b2c-8d95-60a2aa33400b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:45.330536 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef2f67-b7dc-4b2c-8d95-60a2aa33400b None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:45.331321 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef2f67-b7dc-4b2c-8d95-60a2aa33400b None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:45.331805 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef2f67-b7dc-4b2c-8d95-60a2aa33400b None admin] PATH_INFO: `/v3/users/nova` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:45.342985 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef2f67-b7dc-4b2c-8d95-60a2aa33400b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:45.346568 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef2f67-b7dc-4b2c-8d95-60a2aa33400b None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:45.356901 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6a28b-c45f-49a8-aa87-c0625be39862 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:45.377132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-93ef2f67-b7dc-4b2c-8d95-60a2aa33400b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:35:45.377132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:45.377132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:45.377132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:45.377132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:45.377132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:45.377132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:45.377132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:45.377132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:45.377132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:45.377132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:45.377132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:45.377132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:35:45.377132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:35:45.377132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:35:45.377132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:35:45.378201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:45.378201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:45.378201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:35:45.378201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:35:45.378201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:35:45.378201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:35:45.378201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:45.378201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:45.378201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:45.378201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:45.378201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:45.378201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:45.378201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:45.378201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:45.378201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:45.378201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:45.378201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:45.378201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:45.379140 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:45.379140 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:45.379140 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:35:45.379140 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:35:45.379140 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:35:45.379140 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:35:45.379140 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:35:45.379140 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:35:45.379140 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:35:45.379140 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:45.383545 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 246/489] 10.222.0.115 () {56 vars in 1098 bytes} [Tue Aug 2 10:35:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:45.396150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef2f67-b7dc-4b2c-8d95-60a2aa33400b None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:45.488736 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:45.493522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d6a28b-c45f-49a8-aa87-c0625be39862 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:45.534138 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42d14a-4f3f-47cf-bec7-edda28a8880c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:45.535141 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42d14a-4f3f-47cf-bec7-edda28a8880c None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:45.535604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42d14a-4f3f-47cf-bec7-edda28a8880c None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:45.536029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be42d14a-4f3f-47cf-bec7-edda28a8880c None admin] PATH_INFO: `/v3/roles/service` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:45.548529 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be42d14a-4f3f-47cf-bec7-edda28a8880c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:45.553966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-be42d14a-4f3f-47cf-bec7-edda28a8880c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:35:45.553966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:45.553966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:45.553966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:45.553966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:45.553966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:45.553966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:45.553966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:45.553966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:45.553966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:45.553966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:45.553966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:45.553966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:45.553966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:45.553966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:45.553966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:45.555177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:45.555177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:45.555177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:45.555177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:45.555177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:45.555177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:45.555177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:45.555177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:45.555177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:45.555177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:45.555177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:45.555177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:45.555177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:45.555177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:45.555177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:45.555177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:45.556219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:45.556219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:45.556219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:45.556219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:45.556219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:45.556219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:45.556219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:45.556219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:45.556219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:45.556219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:45.556219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:35:45.556219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:45.558587 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 244/490] 10.222.0.115 () {56 vars in 1107 bytes} [Tue Aug 2 10:35:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:35:45.565257 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42d14a-4f3f-47cf-bec7-edda28a8880c None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:45.657447 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef2f67-b7dc-4b2c-8d95-60a2aa33400b None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:45.695605 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d5eaa-79e6-436c-82e5-21639104b18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:45.696236 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d5eaa-79e6-436c-82e5-21639104b18e None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:45.696521 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d5eaa-79e6-436c-82e5-21639104b18e None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:45.696751 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be42d14a-4f3f-47cf-bec7-edda28a8880c None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:45.696852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d5eaa-79e6-436c-82e5-21639104b18e None admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:45.697631 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d5eaa-79e6-436c-82e5-21639104b18e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:45.697941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d5eaa-79e6-436c-82e5-21639104b18e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:45.700490 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d5eaa-79e6-436c-82e5-21639104b18e None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:45.738584 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304853d1-fc2a-411f-9985-f733e0116835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:45.739773 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304853d1-fc2a-411f-9985-f733e0116835 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:45.740594 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304853d1-fc2a-411f-9985-f733e0116835 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:45.740594 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304853d1-fc2a-411f-9985-f733e0116835 None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:45.741053 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304853d1-fc2a-411f-9985-f733e0116835 None admin] RBAC: Adding query filter params (name=service) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:45.741356 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304853d1-fc2a-411f-9985-f733e0116835 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:45.742742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304853d1-fc2a-411f-9985-f733e0116835 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:45.746695 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 247/491] 10.222.0.115 () {56 vars in 1138 bytes} [Tue Aug 2 10:35:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 351 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:45.748207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced497f3-028a-4032-ae55-24bf073a6c35 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:45.748643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced497f3-028a-4032-ae55-24bf073a6c35 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:45.749051 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced497f3-028a-4032-ae55-24bf073a6c35 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:45.755123 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 248/492] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:45 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:45.756469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e24931-6bb8-43c9-a604-88bc05cd7b9a None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:45.759561 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e24931-6bb8-43c9-a604-88bc05cd7b9a None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:45.759767 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 245/493] 10.222.0.115 () {56 vars in 1108 bytes} [Tue Aug 2 10:35:45 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:45.759907 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e24931-6bb8-43c9-a604-88bc05cd7b9a None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:45.760038 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304853d1-fc2a-411f-9985-f733e0116835 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:45.760165 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 249/494] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:45 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:45.760165 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd24a09-4bbf-48b4-9815-8aa0f85ca6d1 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:45.760165 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd24a09-4bbf-48b4-9815-8aa0f85ca6d1 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:45.760165 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd24a09-4bbf-48b4-9815-8aa0f85ca6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:45.782744 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-4fd24a09-4bbf-48b4-9815-8aa0f85ca6d1 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:45.784005 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-4fd24a09-4bbf-48b4-9815-8aa0f85ca6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:45.788574 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:45.812073 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:45.858314 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 250/495] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:45 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:45.863264 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04042ef2-cc8f-4013-b3d6-edc872d3452a None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:45.863264 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04042ef2-cc8f-4013-b3d6-edc872d3452a None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:45.863264 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04042ef2-cc8f-4013-b3d6-edc872d3452a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:45.880364 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-04042ef2-cc8f-4013-b3d6-edc872d3452a None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:45.881698 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-04042ef2-cc8f-4013-b3d6-edc872d3452a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:45.886679 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304853d1-fc2a-411f-9985-f733e0116835 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:45.924845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27bb31e-ffba-4df7-b027-44930b9c3a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:45.925404 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bb31e-ffba-4df7-b027-44930b9c3a4e None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:45.925691 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bb31e-ffba-4df7-b027-44930b9c3a4e None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:45.925974 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27bb31e-ffba-4df7-b027-44930b9c3a4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:45.956557 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 251/496] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:45 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:45.957445 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04042ef2-cc8f-4013-b3d6-edc872d3452a None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:46.020666 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27bb31e-ffba-4df7-b027-44930b9c3a4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:46.022174 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27bb31e-ffba-4df7-b027-44930b9c3a4e None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:46.047893 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:46.063315 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:46.072826 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04042ef2-cc8f-4013-b3d6-edc872d3452a None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:46.110631 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabdbbcb-3080-442d-b995-0e56a7f594e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:46.111329 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabdbbcb-3080-442d-b995-0e56a7f594e0 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:46.111599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabdbbcb-3080-442d-b995-0e56a7f594e0 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:46.113004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabdbbcb-3080-442d-b995-0e56a7f594e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:46.126268 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabdbbcb-3080-442d-b995-0e56a7f594e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:46.127977 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabdbbcb-3080-442d-b995-0e56a7f594e0 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:46.140165 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 246/497] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:35:45 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:35:46.140165 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27bb31e-ffba-4df7-b027-44930b9c3a4e None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:46.145392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-dabdbbcb-3080-442d-b995-0e56a7f594e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:46.145392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:46.145392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:46.145392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:46.145392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:46.145392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:46.145392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:46.145392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:46.145392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:46.145392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:46.145392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:46.145392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:46.145392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:35:46.145392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:35:46.145392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:35:46.146477 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:35:46.146477 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:46.146477 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:46.146477 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:46.146477 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:46.146477 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:46.146477 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:46.146477 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:46.146477 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:46.146477 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:46.146477 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:46.146477 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:46.146477 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:46.146477 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:46.146477 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:46.146477 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:46.146477 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:46.147417 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:35:46.147417 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:35:46.147417 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:46.147417 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:46.147417 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 252/498] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:46.147417 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabdbbcb-3080-442d-b995-0e56a7f594e0 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:46.263659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabdbbcb-3080-442d-b995-0e56a7f594e0 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:46.265121 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27bb31e-ffba-4df7-b027-44930b9c3a4e None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:46.288109 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:46.301180 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080d287-869f-44cc-8c4d-1d897d989eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:46.301818 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080d287-869f-44cc-8c4d-1d897d989eb4 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:46.302096 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080d287-869f-44cc-8c4d-1d897d989eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:46.302368 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080d287-869f-44cc-8c4d-1d897d989eb4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:46.302651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40602c02-96e3-4ce3-942c-551f0f67799c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:46.303395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40602c02-96e3-4ce3-942c-551f0f67799c None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:46.303674 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40602c02-96e3-4ce3-942c-551f0f67799c None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:46.303945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40602c02-96e3-4ce3-942c-551f0f67799c None admin] PATH_INFO: `/v3/roles/service` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:46.313892 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3080d287-869f-44cc-8c4d-1d897d989eb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:46.316615 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40602c02-96e3-4ce3-942c-551f0f67799c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:46.319474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-3080d287-869f-44cc-8c4d-1d897d989eb4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:35:46.319474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:46.319474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:46.319474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:46.319474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:46.319474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:46.319474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:46.319474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:46.319474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:46.319474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:46.319474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:46.319474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:46.319474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:46.319474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:46.319474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:46.319474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:46.320363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:46.320363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:46.320363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:46.320363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:46.320363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:46.320363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:46.320363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:46.320363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:46.320363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:46.320363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:46.320363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:46.320363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:46.320363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:46.320363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:46.320363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:46.320363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:46.321151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:46.321151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:46.321151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:46.321151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:46.321151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:46.321151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:46.321151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:46.321151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:46.321151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:46.321151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:46.321151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:35:46.321151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:46.321151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 253/499] 10.222.0.115 () {56 vars in 1107 bytes} [Tue Aug 2 10:35:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:46.321814 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080d287-869f-44cc-8c4d-1d897d989eb4 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:46.326352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-40602c02-96e3-4ce3-942c-551f0f67799c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:35:46.326352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:46.326352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:46.326352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:46.326352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:46.326352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:46.326352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:46.326352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:46.326352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:46.326352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:46.326352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:46.326352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:46.326352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:46.326352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:46.326352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:46.326352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:46.327303 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:46.327303 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:46.327303 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:46.327303 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:46.327303 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:46.327303 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:46.327303 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:46.327303 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:46.327303 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:46.327303 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:46.327303 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:46.327303 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:46.327303 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:46.327303 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:46.327303 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:46.327303 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:46.328140 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:46.328140 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:46.328140 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:46.328140 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:46.328140 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:46.328140 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:46.328140 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:46.328140 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:46.328140 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:46.328140 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:46.328140 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:35:46.328140 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:46.329109 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:46.329178 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 247/500] 10.222.0.115 () {56 vars in 1107 bytes} [Tue Aug 2 10:35:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:46.329178 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40602c02-96e3-4ce3-942c-551f0f67799c None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:46.427429 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080d287-869f-44cc-8c4d-1d897d989eb4 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:46.433408 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40602c02-96e3-4ce3-942c-551f0f67799c None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:46.462748 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cf195-c593-45b9-a56a-cc3d32b0f968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:46.463417 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cf195-c593-45b9-a56a-cc3d32b0f968 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:46.463706 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cf195-c593-45b9-a56a-cc3d32b0f968 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:46.463979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14cf195-c593-45b9-a56a-cc3d32b0f968 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:46.472310 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfbec3-2fb7-46df-8f8d-7990cb840a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:46.472921 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfbec3-2fb7-46df-8f8d-7990cb840a47 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:46.473278 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfbec3-2fb7-46df-8f8d-7990cb840a47 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:46.473473 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfbec3-2fb7-46df-8f8d-7990cb840a47 None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:46.474162 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bfbec3-2fb7-46df-8f8d-7990cb840a47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:46.474466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bfbec3-2fb7-46df-8f8d-7990cb840a47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:46.475868 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bfbec3-2fb7-46df-8f8d-7990cb840a47 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:46.476062 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cf195-c593-45b9-a56a-cc3d32b0f968 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:46.477554 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14cf195-c593-45b9-a56a-cc3d32b0f968 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:46.488287 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 248/501] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:35:46 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:46.489118 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfbec3-2fb7-46df-8f8d-7990cb840a47 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:46.493243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-e14cf195-c593-45b9-a56a-cc3d32b0f968 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:46.493243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:46.493243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:46.493243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:46.493243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:46.493243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:46.493243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:46.493243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:46.493243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:46.493243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:46.493243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:46.493243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:46.493243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:35:46.493243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:35:46.493243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:35:46.494276 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:35:46.494276 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:46.494276 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:46.494276 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:46.494276 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:46.494276 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:46.494276 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:46.494276 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:46.494276 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:46.494276 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:46.494276 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:46.494276 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:46.494276 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:46.494276 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:46.494276 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:46.494276 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:46.494276 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:46.495166 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:35:46.495166 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:35:46.495166 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:46.495166 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:46.495166 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 254/502] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:46.495427 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cf195-c593-45b9-a56a-cc3d32b0f968 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:46.537707 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:46.554764 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:46.599338 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfbec3-2fb7-46df-8f8d-7990cb840a47 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:46.600162 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14cf195-c593-45b9-a56a-cc3d32b0f968 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:46.634489 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf315459-9599-4ccb-8c4e-2caa8e19585d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:46.635125 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf315459-9599-4ccb-8c4e-2caa8e19585d None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:46.635306 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80182f3c-d70f-4208-955c-2ccc0ea76b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:46.635433 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf315459-9599-4ccb-8c4e-2caa8e19585d None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:46.635769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf315459-9599-4ccb-8c4e-2caa8e19585d None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:46.635944 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80182f3c-d70f-4208-955c-2ccc0ea76b3d None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:46.636224 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80182f3c-d70f-4208-955c-2ccc0ea76b3d None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:46.636489 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf315459-9599-4ccb-8c4e-2caa8e19585d None admin] RBAC: Adding query filter params (name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:46.636594 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80182f3c-d70f-4208-955c-2ccc0ea76b3d None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:46.636794 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf315459-9599-4ccb-8c4e-2caa8e19585d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:46.637205 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80182f3c-d70f-4208-955c-2ccc0ea76b3d None admin] RBAC: Adding query filter params (name=service) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:46.637533 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80182f3c-d70f-4208-955c-2ccc0ea76b3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:46.638225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf315459-9599-4ccb-8c4e-2caa8e19585d None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:46.638892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80182f3c-d70f-4208-955c-2ccc0ea76b3d None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:46.646868 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 255/503] 10.222.0.115 () {56 vars in 1108 bytes} [Tue Aug 2 10:35:46 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:46.651896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 249/504] 10.222.0.115 () {56 vars in 1108 bytes} [Tue Aug 2 10:35:46 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:46.652032 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193b707-f4cf-4be4-95b6-cab3e5048ca9 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:46.652109 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80182f3c-d70f-4208-955c-2ccc0ea76b3d None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:46.652262 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193b707-f4cf-4be4-95b6-cab3e5048ca9 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:46.652262 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193b707-f4cf-4be4-95b6-cab3e5048ca9 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:46.652262 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 256/505] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:46 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:46.652262 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193b707-f4cf-4be4-95b6-cab3e5048ca9 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:46.757326 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80182f3c-d70f-4208-955c-2ccc0ea76b3d None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:46.759016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:46.762633 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193b707-f4cf-4be4-95b6-cab3e5048ca9 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:46.780606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:46.795127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54704a2b-73fe-46e0-bcd8-d3e8f9a84b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:46.795754 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54704a2b-73fe-46e0-bcd8-d3e8f9a84b5d None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:46.796064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54704a2b-73fe-46e0-bcd8-d3e8f9a84b5d None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:46.796645 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54704a2b-73fe-46e0-bcd8-d3e8f9a84b5d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:46.798516 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8a209-355f-48b2-87dd-fe83fe3d2241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:46.799161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8a209-355f-48b2-87dd-fe83fe3d2241 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:46.799443 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8a209-355f-48b2-87dd-fe83fe3d2241 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:46.799715 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e8a209-355f-48b2-87dd-fe83fe3d2241 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:46.800413 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8a209-355f-48b2-87dd-fe83fe3d2241 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:46.800715 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8a209-355f-48b2-87dd-fe83fe3d2241 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:46.802100 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e8a209-355f-48b2-87dd-fe83fe3d2241 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:46.805719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54704a2b-73fe-46e0-bcd8-d3e8f9a84b5d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:46.807590 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54704a2b-73fe-46e0-bcd8-d3e8f9a84b5d None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:46.815210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 257/506] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:35:46 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:46.816100 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8a209-355f-48b2-87dd-fe83fe3d2241 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:46.822786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-54704a2b-73fe-46e0-bcd8-d3e8f9a84b5d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 10:35:46.822786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:46.822786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:46.822786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:46.822786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:46.822786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:46.822786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:46.822786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:46.822786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:46.822786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:46.822786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:46.822786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:46.822786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:35:46.822786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:35:46.822786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:35:46.822786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:35:46.823965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:46.823965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:46.823965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:35:46.823965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:35:46.823965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:35:46.823965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:35:46.823965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:46.823965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:46.823965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:46.823965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:46.823965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:46.823965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:46.823965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:46.823965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:46.823965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:46.823965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:46.823965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:46.823965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:46.825089 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:46.825089 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:46.825089 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:35:46.825089 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:35:46.825089 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:35:46.825089 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:35:46.825089 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:35:46.825089 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:35:46.825089 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 10:35:46.825089 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:46.825089 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 250/507] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:35:46 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:46.825089 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54704a2b-73fe-46e0-bcd8-d3e8f9a84b5d None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:46.925493 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e8a209-355f-48b2-87dd-fe83fe3d2241 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:46.926048 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54704a2b-73fe-46e0-bcd8-d3e8f9a84b5d None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:46.960311 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6af829-c417-4485-a41c-1f51a9b07f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:46.960676 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d077efb-cc48-4b6d-aec2-1a0de0d38b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:46.960937 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6af829-c417-4485-a41c-1f51a9b07f1f None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:46.961213 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6af829-c417-4485-a41c-1f51a9b07f1f None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:46.961305 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d077efb-cc48-4b6d-aec2-1a0de0d38b9a None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:46.961484 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6af829-c417-4485-a41c-1f51a9b07f1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:46.961602 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d077efb-cc48-4b6d-aec2-1a0de0d38b9a None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:46.961832 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d077efb-cc48-4b6d-aec2-1a0de0d38b9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:46.972188 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6af829-c417-4485-a41c-1f51a9b07f1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:46.972339 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d077efb-cc48-4b6d-aec2-1a0de0d38b9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:46.973665 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6af829-c417-4485-a41c-1f51a9b07f1f None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:46.973794 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d077efb-cc48-4b6d-aec2-1a0de0d38b9a None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:46.988932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-0e6af829-c417-4485-a41c-1f51a9b07f1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:46.988932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:46.988932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:46.988932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:46.988932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:46.988932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:46.988932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:46.988932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:46.988932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:46.988932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:46.988932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:46.988932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:46.988932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:35:46.988932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:35:46.988932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:35:46.989880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:35:46.989880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:46.989880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:46.989880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:46.989880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:46.989880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:46.989880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:46.989880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:46.989880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:46.989880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:46.989880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:46.989880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:46.989880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:46.989880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:46.989880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:46.989880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:46.989880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:46.995678 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:35:46.995678 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:35:46.995678 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:46.995678 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:46.995948 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-4d077efb-cc48-4b6d-aec2-1a0de0d38b9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:46.995948 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:46.995948 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:46.995948 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:46.995948 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:46.995948 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:46.995948 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:46.995948 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:46.995948 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:46.995948 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:46.995948 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:46.995948 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:46.995948 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:35:46.995948 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:35:46.995948 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:35:46.996939 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:35:46.996939 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:46.996939 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:46.996939 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:46.996939 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:46.996939 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:46.996939 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:46.996939 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:46.996939 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:46.996939 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:46.996939 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:46.996939 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:46.996939 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:46.996939 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:46.996939 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:46.996939 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:46.996939 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:46.998070 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:35:46.998070 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:35:46.998070 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:46.998070 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:46.998335 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 251/508] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:46.998417 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 258/509] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:46.998417 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d077efb-cc48-4b6d-aec2-1a0de0d38b9a None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:46.998572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e87776-a187-4a47-b3d1-284090fffd6f None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:46.998572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e87776-a187-4a47-b3d1-284090fffd6f None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:46.998572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e87776-a187-4a47-b3d1-284090fffd6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:47.010533 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-a0e87776-a187-4a47-b3d1-284090fffd6f None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:47.011758 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-a0e87776-a187-4a47-b3d1-284090fffd6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:47.020939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:47.029398 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:47.091155 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 252/510] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:46 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:47.091155 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e87776-a187-4a47-b3d1-284090fffd6f None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:47.100057 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d077efb-cc48-4b6d-aec2-1a0de0d38b9a None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:47.135873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17644dfb-3e89-4463-a2d4-6bc86391aa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:47.136512 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17644dfb-3e89-4463-a2d4-6bc86391aa4d None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:47.137176 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17644dfb-3e89-4463-a2d4-6bc86391aa4d None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:47.137176 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17644dfb-3e89-4463-a2d4-6bc86391aa4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:47.145722 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17644dfb-3e89-4463-a2d4-6bc86391aa4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:47.147227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17644dfb-3e89-4463-a2d4-6bc86391aa4d None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:47.161914 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-17644dfb-3e89-4463-a2d4-6bc86391aa4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:35:47.161914 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:47.161914 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:47.161914 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:47.161914 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:47.161914 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:47.161914 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:47.161914 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:47.161914 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:47.161914 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:47.161914 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:47.161914 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:47.161914 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:35:47.161914 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:35:47.161914 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:35:47.162848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:35:47.162848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:47.162848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:47.162848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:47.162848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:47.162848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:47.162848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:47.162848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:47.162848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:47.162848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:47.162848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:47.162848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:47.162848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:47.162848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:47.162848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:47.162848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:47.162848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:47.164374 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:35:47.164374 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:35:47.164374 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:35:47.164374 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:35:47.164374 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:35:47.164374 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:35:47.164374 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:35:47.164374 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:47.164374 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 259/511] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:35:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:47.164374 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17644dfb-3e89-4463-a2d4-6bc86391aa4d None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:47.192223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e87776-a187-4a47-b3d1-284090fffd6f None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:47.227677 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ca221-3afc-44da-9156-9219588a7c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:47.228289 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ca221-3afc-44da-9156-9219588a7c83 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:47.228565 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ca221-3afc-44da-9156-9219588a7c83 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:47.228835 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ca221-3afc-44da-9156-9219588a7c83 None admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:47.229627 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ca221-3afc-44da-9156-9219588a7c83 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:47.229928 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ca221-3afc-44da-9156-9219588a7c83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:47.231384 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485ca221-3afc-44da-9156-9219588a7c83 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:47.262563 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:47.268577 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 253/512] 10.222.0.115 () {56 vars in 1142 bytes} [Tue Aug 2 10:35:47 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:47.269459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ca221-3afc-44da-9156-9219588a7c83 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:47.272202 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:47.273852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17644dfb-3e89-4463-a2d4-6bc86391aa4d None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:47.308565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282f13c-497b-4cc1-afcf-57be55a051d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:47.309167 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282f13c-497b-4cc1-afcf-57be55a051d3 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:47.309451 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282f13c-497b-4cc1-afcf-57be55a051d3 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:47.309720 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1282f13c-497b-4cc1-afcf-57be55a051d3 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:47.310406 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1282f13c-497b-4cc1-afcf-57be55a051d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:47.310704 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1282f13c-497b-4cc1-afcf-57be55a051d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:47.312808 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1282f13c-497b-4cc1-afcf-57be55a051d3 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:47.324260 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 260/513] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:35:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:47.325064 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282f13c-497b-4cc1-afcf-57be55a051d3 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:47.370856 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ca221-3afc-44da-9156-9219588a7c83 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:47.404380 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4326c7-dd6b-4328-853e-527b6e85586e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:47.404997 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4326c7-dd6b-4328-853e-527b6e85586e None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:47.405273 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4326c7-dd6b-4328-853e-527b6e85586e None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:47.406119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4326c7-dd6b-4328-853e-527b6e85586e None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:47.406915 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4326c7-dd6b-4328-853e-527b6e85586e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:47.407249 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4326c7-dd6b-4328-853e-527b6e85586e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:47.408674 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4326c7-dd6b-4328-853e-527b6e85586e None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:47.420541 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 254/514] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:35:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:47.421998 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4326c7-dd6b-4328-853e-527b6e85586e None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:47.428171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1282f13c-497b-4cc1-afcf-57be55a051d3 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:47.464105 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3265d-e37e-4edc-a3f2-d380fc0cb536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:47.464739 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3265d-e37e-4edc-a3f2-d380fc0cb536 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:47.465022 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3265d-e37e-4edc-a3f2-d380fc0cb536 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:47.465295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f3265d-e37e-4edc-a3f2-d380fc0cb536 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:47.476746 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f3265d-e37e-4edc-a3f2-d380fc0cb536 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:47.478208 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f3265d-e37e-4edc-a3f2-d380fc0cb536 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:47.490923 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:47.493172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-61f3265d-e37e-4edc-a3f2-d380fc0cb536 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:47.493172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:47.493172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:47.493172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:47.493172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:47.493172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:47.493172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:47.493172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:47.493172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:47.493172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:47.493172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:47.493172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:47.493172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:35:47.493172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:35:47.493172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:35:47.497214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:35:47.497214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:47.497214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:47.497214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:47.497214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:47.497214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:47.497214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:47.497214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:47.497214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:47.497214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:47.497214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:47.497214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:47.497214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:47.497214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:47.497214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:47.497214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:47.497214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:47.498076 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:35:47.498076 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:35:47.498076 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:47.498076 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:47.498076 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 261/515] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:47.498076 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3265d-e37e-4edc-a3f2-d380fc0cb536 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:47.502511 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:47.531522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4326c7-dd6b-4328-853e-527b6e85586e None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:47.569496 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407970a0-add6-4833-b7b8-1b6f6a5a514d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:47.570154 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407970a0-add6-4833-b7b8-1b6f6a5a514d None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:47.570435 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407970a0-add6-4833-b7b8-1b6f6a5a514d None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:47.570708 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407970a0-add6-4833-b7b8-1b6f6a5a514d None admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:47.571534 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407970a0-add6-4833-b7b8-1b6f6a5a514d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:47.571878 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407970a0-add6-4833-b7b8-1b6f6a5a514d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:47.573422 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407970a0-add6-4833-b7b8-1b6f6a5a514d None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:47.591879 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 255/516] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:35:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:47.592704 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407970a0-add6-4833-b7b8-1b6f6a5a514d None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:47.606563 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f3265d-e37e-4edc-a3f2-d380fc0cb536 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:47.643729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a177498-470d-458d-a3ae-034dc53e2801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:47.644257 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a177498-470d-458d-a3ae-034dc53e2801 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:47.644537 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a177498-470d-458d-a3ae-034dc53e2801 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:47.644806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a177498-470d-458d-a3ae-034dc53e2801 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:47.703750 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407970a0-add6-4833-b7b8-1b6f6a5a514d None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:47.718587 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a177498-470d-458d-a3ae-034dc53e2801 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:47.720174 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a177498-470d-458d-a3ae-034dc53e2801 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:47.722048 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:47.727778 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:47.739957 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca1c33-e697-45e3-8cca-996d82a9d4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:47.740617 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca1c33-e697-45e3-8cca-996d82a9d4b0 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:47.740896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca1c33-e697-45e3-8cca-996d82a9d4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:47.741170 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca1c33-e697-45e3-8cca-996d82a9d4b0 None admin] PATH_INFO: `/v3/users/glance` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:47.748481 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca1c33-e697-45e3-8cca-996d82a9d4b0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:47.750257 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca1c33-e697-45e3-8cca-996d82a9d4b0 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:47.766882 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-17ca1c33-e697-45e3-8cca-996d82a9d4b0 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 02 10:35:47.766882 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:47.766882 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:47.766882 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:47.766882 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:47.766882 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:47.766882 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:47.766882 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:47.766882 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:47.766882 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:47.766882 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:47.766882 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:47.766882 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:35:47.766882 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:35:47.766882 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:35:47.766882 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:35:47.768059 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:47.768059 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:47.768059 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:35:47.768059 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:35:47.768059 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:35:47.768059 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:35:47.768059 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:47.768059 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:47.768059 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:47.768059 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:47.768059 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:47.768059 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:47.768059 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:47.768059 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:47.768059 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:47.768059 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:47.768059 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:47.768059 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:47.769152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:47.769152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:47.769152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:35:47.769152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:35:47.769152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:35:47.769152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:35:47.769152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:35:47.769152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:35:47.769152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 02 10:35:47.769152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:47.769152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 256/517] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:35:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:47.769765 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca1c33-e697-45e3-8cca-996d82a9d4b0 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:47.830271 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 262/518] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:35:47 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:35:47.831445 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a177498-470d-458d-a3ae-034dc53e2801 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:47.875555 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca1c33-e697-45e3-8cca-996d82a9d4b0 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:47.910893 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48804acc-8638-4240-bf36-d96fd2a52a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:47.911484 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48804acc-8638-4240-bf36-d96fd2a52a3c None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:47.911766 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48804acc-8638-4240-bf36-d96fd2a52a3c None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:47.912045 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48804acc-8638-4240-bf36-d96fd2a52a3c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:47.923349 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48804acc-8638-4240-bf36-d96fd2a52a3c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:47.924988 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48804acc-8638-4240-bf36-d96fd2a52a3c None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:47.933995 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a177498-470d-458d-a3ae-034dc53e2801 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:47.935632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:47.936550 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-48804acc-8638-4240-bf36-d96fd2a52a3c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 10:35:47.936550 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:47.936550 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:47.936550 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:47.936550 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:47.936550 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:47.936550 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:47.936550 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:47.936550 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:47.936550 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:47.936550 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:47.936550 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:47.936550 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:35:47.936550 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:35:47.936550 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:35:47.936550 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:35:47.937586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:47.937586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:47.937586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:35:47.937586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:35:47.937586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:35:47.937586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:35:47.937586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:47.937586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:47.937586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:47.937586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:47.937586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:47.937586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:47.937586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:47.937586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:47.937586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:47.937586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:47.937586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:47.937586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:47.938725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:47.938725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:47.938725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:35:47.938725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:35:47.938725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:35:47.938725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:35:47.938725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:35:47.938725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:35:47.938725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 10:35:47.938725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:47.938725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 257/519] 10.222.0.115 () {56 vars in 1107 bytes} [Tue Aug 2 10:35:47 2022] GET /identity/v3/users/neutron => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:47.938725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48804acc-8638-4240-bf36-d96fd2a52a3c None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:47.975346 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d3198-d29c-41c3-8d2b-12e5959b2ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:47.975983 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d3198-d29c-41c3-8d2b-12e5959b2ce6 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:47.976273 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d3198-d29c-41c3-8d2b-12e5959b2ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:47.976549 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5d3198-d29c-41c3-8d2b-12e5959b2ce6 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:47.977244 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d3198-d29c-41c3-8d2b-12e5959b2ce6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:47.977550 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d3198-d29c-41c3-8d2b-12e5959b2ce6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:47.979005 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5d3198-d29c-41c3-8d2b-12e5959b2ce6 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:47.981859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:47.990313 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 263/520] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:35:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:47.993356 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d3198-d29c-41c3-8d2b-12e5959b2ce6 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:48.041957 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48804acc-8638-4240-bf36-d96fd2a52a3c None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:48.076395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79314c-0f60-4541-9030-8a52b8563f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:48.077004 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79314c-0f60-4541-9030-8a52b8563f1c None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:48.077285 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79314c-0f60-4541-9030-8a52b8563f1c None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:48.078084 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce79314c-0f60-4541-9030-8a52b8563f1c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:48.078861 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce79314c-0f60-4541-9030-8a52b8563f1c None admin] RBAC: Adding query filter params (role.id=0f1c61dab22c4d97ae784468058036c7, scope.project.id=c1eae881e5714bf4bd223809e1a2d920, user.id=d0478bd540f446ec810e2091c46ac49c) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:48.079200 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce79314c-0f60-4541-9030-8a52b8563f1c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:48.080902 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce79314c-0f60-4541-9030-8a52b8563f1c None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:48.086600 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 258/521] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:35:47 2022] GET /identity/v3/role_assignments?user.id=d0478bd540f446ec810e2091c46ac49c&scope.project.id=c1eae881e5714bf4bd223809e1a2d920&role.id=0f1c61dab22c4d97ae784468058036c7 => generated 260 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:48.087445 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79314c-0f60-4541-9030-8a52b8563f1c None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:48.105422 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5d3198-d29c-41c3-8d2b-12e5959b2ce6 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:48.152888 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a755fc0-589f-4bc7-b58f-5d841602042a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:48.153526 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a755fc0-589f-4bc7-b58f-5d841602042a None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:48.153815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a755fc0-589f-4bc7-b58f-5d841602042a None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:48.154074 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a755fc0-589f-4bc7-b58f-5d841602042a None admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:48.154897 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a755fc0-589f-4bc7-b58f-5d841602042a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:48.155232 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a755fc0-589f-4bc7-b58f-5d841602042a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:48.157080 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a755fc0-589f-4bc7-b58f-5d841602042a None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:48.177599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 264/522] 10.222.0.115 () {56 vars in 1142 bytes} [Tue Aug 2 10:35:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:48.182842 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a755fc0-589f-4bc7-b58f-5d841602042a None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:48.209402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce79314c-0f60-4541-9030-8a52b8563f1c None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:48.210102 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:48.217414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:48.252787 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32feadda-c2d8-4691-98a5-c0069c1bc93e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:48.253435 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32feadda-c2d8-4691-98a5-c0069c1bc93e None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:48.253706 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32feadda-c2d8-4691-98a5-c0069c1bc93e None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:48.253973 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32feadda-c2d8-4691-98a5-c0069c1bc93e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:48.266642 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32feadda-c2d8-4691-98a5-c0069c1bc93e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:48.267848 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32feadda-c2d8-4691-98a5-c0069c1bc93e None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:48.283374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-32feadda-c2d8-4691-98a5-c0069c1bc93e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:48.283374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:48.283374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:48.283374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:48.283374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:48.283374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:48.283374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:48.283374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:48.283374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:48.283374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:48.283374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:48.283374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:48.283374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:35:48.283374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:35:48.283374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:35:48.285121 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:35:48.285121 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:48.285121 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:48.285121 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:48.285121 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:48.285121 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:48.285121 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:48.285121 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:48.285121 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:48.285121 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:48.285121 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:48.285121 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:48.285121 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:48.285121 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:48.285121 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:48.285121 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:48.285121 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:48.286896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:35:48.286896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:35:48.286896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:48.286896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:48.286896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 259/523] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:48.286896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32feadda-c2d8-4691-98a5-c0069c1bc93e None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:48.309447 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a755fc0-589f-4bc7-b58f-5d841602042a None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:48.349306 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b412885-b145-40d6-9112-4e3771f992c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:48.349923 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b412885-b145-40d6-9112-4e3771f992c4 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:48.350198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b412885-b145-40d6-9112-4e3771f992c4 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:48.350466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b412885-b145-40d6-9112-4e3771f992c4 None admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:48.351323 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b412885-b145-40d6-9112-4e3771f992c4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:48.351615 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b412885-b145-40d6-9112-4e3771f992c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:48.353039 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b412885-b145-40d6-9112-4e3771f992c4 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:48.372006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 265/524] 10.222.0.115 () {56 vars in 1144 bytes} [Tue Aug 2 10:35:48 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:48.372724 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b412885-b145-40d6-9112-4e3771f992c4 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:48.413651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32feadda-c2d8-4691-98a5-c0069c1bc93e None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:48.456365 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332667cc-8c6d-4623-a2bf-35330bdbf715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:48.457279 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332667cc-8c6d-4623-a2bf-35330bdbf715 None admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:48.457279 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332667cc-8c6d-4623-a2bf-35330bdbf715 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:48.457457 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332667cc-8c6d-4623-a2bf-35330bdbf715 None admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:48.458011 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332667cc-8c6d-4623-a2bf-35330bdbf715 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:48.459491 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332667cc-8c6d-4623-a2bf-35330bdbf715 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:48.471445 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:48.494004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b412885-b145-40d6-9112-4e3771f992c4 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:48.540139 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0678f615-c828-4bac-afc8-0c0b33f9f4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:48.540772 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678f615-c828-4bac-afc8-0c0b33f9f4a1 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:48.541108 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678f615-c828-4bac-afc8-0c0b33f9f4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:48.541322 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0678f615-c828-4bac-afc8-0c0b33f9f4a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:48.568057 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 260/525] 10.222.0.115 () {60 vars in 1137 bytes} [Tue Aug 2 10:35:48 2022] POST /identity/v3/users => generated 248 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:35:48.569359 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332667cc-8c6d-4623-a2bf-35330bdbf715 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:48.590799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:48.651933 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0678f615-c828-4bac-afc8-0c0b33f9f4a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:48.653983 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0678f615-c828-4bac-afc8-0c0b33f9f4a1 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:48.722241 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332667cc-8c6d-4623-a2bf-35330bdbf715 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:48.765410 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99b318-803d-4112-b02d-7a1456a5d665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:48.766253 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:48.767188 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99b318-803d-4112-b02d-7a1456a5d665 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:48.767635 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99b318-803d-4112-b02d-7a1456a5d665 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:48.768058 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99b318-803d-4112-b02d-7a1456a5d665 None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:48.768925 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe99b318-803d-4112-b02d-7a1456a5d665 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:48.769370 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe99b318-803d-4112-b02d-7a1456a5d665 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:48.770901 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe99b318-803d-4112-b02d-7a1456a5d665 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:48.789227 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 261/526] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:35:48 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:48.793911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99b318-803d-4112-b02d-7a1456a5d665 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:48.806792 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 266/527] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:35:48 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:35:48.808808 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0678f615-c828-4bac-afc8-0c0b33f9f4a1 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:48.919451 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:48.932616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99b318-803d-4112-b02d-7a1456a5d665 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:48.938152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0678f615-c828-4bac-afc8-0c0b33f9f4a1 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:48.972414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6316c81d-313b-43ec-bf0b-47101aad3a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:48.973272 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316c81d-313b-43ec-bf0b-47101aad3a03 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:48.973740 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316c81d-313b-43ec-bf0b-47101aad3a03 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:48.974164 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6316c81d-313b-43ec-bf0b-47101aad3a03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:48.975647 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa13427-8d37-4c85-a660-7c205d2dc8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:48.976363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa13427-8d37-4c85-a660-7c205d2dc8c9 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:48.976713 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa13427-8d37-4c85-a660-7c205d2dc8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:48.977055 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa13427-8d37-4c85-a660-7c205d2dc8c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:48.989963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa13427-8d37-4c85-a660-7c205d2dc8c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:48.992307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa13427-8d37-4c85-a660-7c205d2dc8c9 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:49.009006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-5fa13427-8d37-4c85-a660-7c205d2dc8c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:35:49.009006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:49.009006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:49.009006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:49.009006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:49.009006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:49.009006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:49.009006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:49.009006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:49.009006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:49.009006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:49.009006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:49.009006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:35:49.009006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:35:49.009006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:35:49.010095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:35:49.010095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:49.010095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:49.010095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:49.010095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:49.010095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:49.010095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:49.010095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:49.010095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:49.010095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:49.010095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:49.010095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:49.010095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:49.010095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:49.010095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:49.010095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:49.010095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:49.011200 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:35:49.011200 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:35:49.011200 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:35:49.011200 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:35:49.011200 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:35:49.011200 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:35:49.011200 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:35:49.011200 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:49.012480 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 267/528] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:35:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:49.018823 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa13427-8d37-4c85-a660-7c205d2dc8c9 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:49.054505 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6316c81d-313b-43ec-bf0b-47101aad3a03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:49.056719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6316c81d-313b-43ec-bf0b-47101aad3a03 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:49.073335 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:49.153863 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa13427-8d37-4c85-a660-7c205d2dc8c9 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:49.155167 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:49.192597 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 262/529] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:35:48 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:35:49.192597 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6316c81d-313b-43ec-bf0b-47101aad3a03 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:49.208397 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d109d86-83a5-4f1c-b0ba-9812e32a21d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:49.210779 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d109d86-83a5-4f1c-b0ba-9812e32a21d9 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:49.211495 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d109d86-83a5-4f1c-b0ba-9812e32a21d9 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:49.211926 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d109d86-83a5-4f1c-b0ba-9812e32a21d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:49.228875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d109d86-83a5-4f1c-b0ba-9812e32a21d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:49.231522 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d109d86-83a5-4f1c-b0ba-9812e32a21d9 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:49.253509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-7d109d86-83a5-4f1c-b0ba-9812e32a21d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:49.253509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:49.253509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:49.253509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:49.253509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:49.253509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:49.253509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:49.253509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:49.253509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:49.253509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:49.253509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:49.253509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:49.253509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:35:49.253509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:35:49.253509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:35:49.254544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:35:49.254544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:49.254544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:49.254544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:49.254544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:49.254544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:49.254544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:49.254544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:49.254544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:49.254544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:49.254544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:49.254544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:49.254544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:49.254544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:49.254544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:49.254544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:49.254544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:49.255442 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:35:49.255442 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:35:49.255442 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:49.255442 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:49.257855 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 268/530] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:49.258821 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d109d86-83a5-4f1c-b0ba-9812e32a21d9 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:49.322146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6316c81d-313b-43ec-bf0b-47101aad3a03 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:49.368411 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4063a14c-eb61-403d-aff9-222d080e95ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:49.368411 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063a14c-eb61-403d-aff9-222d080e95ef None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:49.368411 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063a14c-eb61-403d-aff9-222d080e95ef None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:49.368411 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4063a14c-eb61-403d-aff9-222d080e95ef None admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:49.368892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4063a14c-eb61-403d-aff9-222d080e95ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:49.368892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4063a14c-eb61-403d-aff9-222d080e95ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:49.368892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4063a14c-eb61-403d-aff9-222d080e95ef None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:49.371217 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:49.382243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d109d86-83a5-4f1c-b0ba-9812e32a21d9 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:49.398959 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 263/531] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:35:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:49.398959 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4063a14c-eb61-403d-aff9-222d080e95ef None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:49.421521 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:49.428378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec95827-207d-482c-88f5-ea77893d31f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:49.434633 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec95827-207d-482c-88f5-ea77893d31f5 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:49.434633 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec95827-207d-482c-88f5-ea77893d31f5 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:49.434633 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec95827-207d-482c-88f5-ea77893d31f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:49.446458 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec95827-207d-482c-88f5-ea77893d31f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:49.448206 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec95827-207d-482c-88f5-ea77893d31f5 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:49.471280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-4ec95827-207d-482c-88f5-ea77893d31f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:49.471280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:49.471280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:49.471280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:49.471280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:49.471280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:49.471280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:49.471280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:49.471280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:49.471280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:49.471280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:49.471280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:49.471280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:35:49.471280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:35:49.471280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:35:49.472243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:35:49.472243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:49.472243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:49.472243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:49.472243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:49.472243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:49.472243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:49.472243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:49.472243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:49.472243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:49.472243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:49.472243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:49.472243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:49.472243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:49.472243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:49.472243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:49.472243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:49.473097 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:35:49.473097 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:35:49.473097 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:49.473097 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:49.474536 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 269/532] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:49.475773 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec95827-207d-482c-88f5-ea77893d31f5 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:49.509852 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4063a14c-eb61-403d-aff9-222d080e95ef None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:49.613131 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec95827-207d-482c-88f5-ea77893d31f5 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:49.616451 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851806e-c168-4326-8006-e86d178b2145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:49.618555 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851806e-c168-4326-8006-e86d178b2145 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:49.619034 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851806e-c168-4326-8006-e86d178b2145 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:49.619510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d851806e-c168-4326-8006-e86d178b2145 None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:49.620407 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851806e-c168-4326-8006-e86d178b2145 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:49.620921 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851806e-c168-4326-8006-e86d178b2145 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:49.622483 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d851806e-c168-4326-8006-e86d178b2145 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:49.643170 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 264/533] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:35:49 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:49.645116 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851806e-c168-4326-8006-e86d178b2145 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:49.663790 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a055e48-9c5f-4547-a8f8-b53dd67652e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:49.664737 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a055e48-9c5f-4547-a8f8-b53dd67652e2 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:49.665202 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a055e48-9c5f-4547-a8f8-b53dd67652e2 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:49.665658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a055e48-9c5f-4547-a8f8-b53dd67652e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:49.666901 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a055e48-9c5f-4547-a8f8-b53dd67652e2 None admin] RBAC: Adding query filter params (role.id=0f1c61dab22c4d97ae784468058036c7, scope.project.id=c1eae881e5714bf4bd223809e1a2d920, user.id=cc9ab3c9075b4a8385ef634dc0b1625e) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:49.667480 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a055e48-9c5f-4547-a8f8-b53dd67652e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:49.669190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a055e48-9c5f-4547-a8f8-b53dd67652e2 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:49.683234 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 270/534] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:35:49 2022] GET /identity/v3/role_assignments?user.id=cc9ab3c9075b4a8385ef634dc0b1625e&scope.project.id=c1eae881e5714bf4bd223809e1a2d920&role.id=0f1c61dab22c4d97ae784468058036c7 => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:49.684542 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a055e48-9c5f-4547-a8f8-b53dd67652e2 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:49.699573 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:49.784255 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:49.792668 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d851806e-c168-4326-8006-e86d178b2145 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:49.834643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858e46f-9723-43f4-8266-ad3b6c1f827b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:49.835611 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858e46f-9723-43f4-8266-ad3b6c1f827b None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:49.836229 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858e46f-9723-43f4-8266-ad3b6c1f827b None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:49.836692 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2858e46f-9723-43f4-8266-ad3b6c1f827b None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:49.837581 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2858e46f-9723-43f4-8266-ad3b6c1f827b None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:49.838064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2858e46f-9723-43f4-8266-ad3b6c1f827b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:49.839655 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2858e46f-9723-43f4-8266-ad3b6c1f827b None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:49.854450 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 265/535] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:35:49 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:49.862721 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858e46f-9723-43f4-8266-ad3b6c1f827b None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:49.907027 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a055e48-9c5f-4547-a8f8-b53dd67652e2 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:49.987539 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2858e46f-9723-43f4-8266-ad3b6c1f827b None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:49.988544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470431a0-d5d9-4db9-a664-613e7d38d9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:49.989192 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470431a0-d5d9-4db9-a664-613e7d38d9cd None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:49.989468 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470431a0-d5d9-4db9-a664-613e7d38d9cd None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:49.989738 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470431a0-d5d9-4db9-a664-613e7d38d9cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:50.002580 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470431a0-d5d9-4db9-a664-613e7d38d9cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:50.004059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470431a0-d5d9-4db9-a664-613e7d38d9cd None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:50.018784 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-470431a0-d5d9-4db9-a664-613e7d38d9cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:35:50.018784 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:50.018784 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:50.018784 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:50.018784 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:50.018784 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:50.018784 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:50.018784 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:50.018784 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:50.018784 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:50.018784 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:50.018784 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:50.018784 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:35:50.018784 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:35:50.018784 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:35:50.019761 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:35:50.019761 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:50.019761 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:50.019761 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:50.019761 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:50.019761 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:50.019761 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:50.019761 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:50.019761 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:50.019761 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:50.019761 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:50.019761 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:50.019761 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:50.019761 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:50.019761 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:50.019761 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:50.019761 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:50.020628 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:35:50.020628 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:35:50.020628 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:35:50.020628 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:35:50.020628 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:35:50.020628 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:35:50.020628 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:35:50.020628 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:50.020628 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 271/536] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:35:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:50.027352 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470431a0-d5d9-4db9-a664-613e7d38d9cd None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:50.031221 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7601d5-a6d0-46b3-984c-908afa4adc87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:50.032229 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7601d5-a6d0-46b3-984c-908afa4adc87 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:50.032709 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7601d5-a6d0-46b3-984c-908afa4adc87 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:50.033270 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7601d5-a6d0-46b3-984c-908afa4adc87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:50.041168 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7601d5-a6d0-46b3-984c-908afa4adc87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:50.042585 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7601d5-a6d0-46b3-984c-908afa4adc87 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:50.065796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-0e7601d5-a6d0-46b3-984c-908afa4adc87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:35:50.065796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:50.065796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:50.065796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:50.065796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:50.065796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:50.065796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:50.065796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:50.065796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:50.065796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:50.065796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:50.065796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:50.065796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:35:50.065796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:35:50.065796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:35:50.066719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:35:50.066719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:50.066719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:50.066719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:50.066719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:50.066719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:50.066719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:50.066719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:50.066719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:50.066719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:50.066719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:50.066719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:50.066719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:50.066719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:50.066719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:50.066719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:50.066719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:50.067602 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:35:50.067602 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:35:50.067602 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:35:50.067602 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:35:50.067602 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:35:50.067602 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:35:50.067602 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:35:50.067602 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:50.067602 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 266/537] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:35:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:50.074169 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7601d5-a6d0-46b3-984c-908afa4adc87 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:50.095414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:50.170875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:50.188245 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7601d5-a6d0-46b3-984c-908afa4adc87 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:50.236540 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e23e47-5d60-4142-9c0d-2f2984d3881a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:50.237267 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e23e47-5d60-4142-9c0d-2f2984d3881a None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:50.237620 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e23e47-5d60-4142-9c0d-2f2984d3881a None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:50.237957 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e23e47-5d60-4142-9c0d-2f2984d3881a None admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:50.238789 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e23e47-5d60-4142-9c0d-2f2984d3881a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:50.239196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e23e47-5d60-4142-9c0d-2f2984d3881a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:50.241402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e23e47-5d60-4142-9c0d-2f2984d3881a None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:50.257168 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 267/538] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:35:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:50.262931 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e23e47-5d60-4142-9c0d-2f2984d3881a None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:50.275195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470431a0-d5d9-4db9-a664-613e7d38d9cd None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:50.312480 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1331002-b172-4aa5-80cb-d041643e1958 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:50.313022 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1331002-b172-4aa5-80cb-d041643e1958 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:50.313305 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1331002-b172-4aa5-80cb-d041643e1958 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:50.313581 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1331002-b172-4aa5-80cb-d041643e1958 None admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:50.314344 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1331002-b172-4aa5-80cb-d041643e1958 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:50.314653 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1331002-b172-4aa5-80cb-d041643e1958 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:50.316196 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1331002-b172-4aa5-80cb-d041643e1958 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:50.336222 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 272/539] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:35:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:50.337576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903a815-32af-434f-8060-eaec6b49c11a None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:50.337871 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903a815-32af-434f-8060-eaec6b49c11a None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:50.338152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903a815-32af-434f-8060-eaec6b49c11a None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:50.344215 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 273/540] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:50 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:50.345121 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903a815-32af-434f-8060-eaec6b49c11a None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:50.359490 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:50.383738 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e23e47-5d60-4142-9c0d-2f2984d3881a None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:50.422639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c7852-d1a1-4866-8772-45addb6d080a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:50.423288 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c7852-d1a1-4866-8772-45addb6d080a None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:50.423567 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c7852-d1a1-4866-8772-45addb6d080a None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:50.423840 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c7852-d1a1-4866-8772-45addb6d080a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:50.424631 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c7852-d1a1-4866-8772-45addb6d080a None admin] RBAC: Adding query filter params (role.id=0f1c61dab22c4d97ae784468058036c7, scope.project.id=c1eae881e5714bf4bd223809e1a2d920, user.id=98dcb74152ec4375a586ea526e7f07c8) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:50.425043 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c7852-d1a1-4866-8772-45addb6d080a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:50.426625 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c7852-d1a1-4866-8772-45addb6d080a None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:50.443073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 268/541] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:35:50 2022] GET /identity/v3/role_assignments?user.id=98dcb74152ec4375a586ea526e7f07c8&scope.project.id=c1eae881e5714bf4bd223809e1a2d920&role.id=0f1c61dab22c4d97ae784468058036c7 => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:50.443073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792f271-39d3-424b-a9f9-5ca4d884c65c None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:50.443073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792f271-39d3-424b-a9f9-5ca4d884c65c None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:50.443073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792f271-39d3-424b-a9f9-5ca4d884c65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:50.465744 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903a815-32af-434f-8060-eaec6b49c11a None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:50.487382 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-7792f271-39d3-424b-a9f9-5ca4d884c65c None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:50.488721 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-7792f271-39d3-424b-a9f9-5ca4d884c65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:50.500904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:50.515933 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d082e99-5d76-4fd9-aae5-b7701affcb49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:50.516623 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d082e99-5d76-4fd9-aae5-b7701affcb49 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:50.516907 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d082e99-5d76-4fd9-aae5-b7701affcb49 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:50.517183 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d082e99-5d76-4fd9-aae5-b7701affcb49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:50.517994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d082e99-5d76-4fd9-aae5-b7701affcb49 None admin] RBAC: Adding query filter params (role.id=0f1c61dab22c4d97ae784468058036c7, scope.project.id=c1eae881e5714bf4bd223809e1a2d920, user.id=e358fd7c1b3c43d7ac89fb3562678266) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:50.518407 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d082e99-5d76-4fd9-aae5-b7701affcb49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:50.519974 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d082e99-5d76-4fd9-aae5-b7701affcb49 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:50.538137 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 274/542] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:35:50 2022] GET /identity/v3/role_assignments?user.id=e358fd7c1b3c43d7ac89fb3562678266&scope.project.id=c1eae881e5714bf4bd223809e1a2d920&role.id=0f1c61dab22c4d97ae784468058036c7 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:50.587841 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 269/543] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:50 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:50.619660 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d082e99-5d76-4fd9-aae5-b7701affcb49 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:50.755941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d082e99-5d76-4fd9-aae5-b7701affcb49 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:50.794230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44718847-e4bd-49da-ad1c-7c104af7aefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:50.794875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44718847-e4bd-49da-ad1c-7c104af7aefa None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:50.795186 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44718847-e4bd-49da-ad1c-7c104af7aefa None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:50.795460 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44718847-e4bd-49da-ad1c-7c104af7aefa None admin] PATH_INFO: `/v3/roles/service` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:50.806908 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44718847-e4bd-49da-ad1c-7c104af7aefa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:50.812775 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-44718847-e4bd-49da-ad1c-7c104af7aefa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:35:50.812775 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:50.812775 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:50.812775 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:50.812775 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:50.812775 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:50.812775 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:50.812775 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:50.812775 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:50.812775 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:50.812775 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:50.812775 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:50.812775 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:50.812775 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:50.812775 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:50.812775 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:50.813683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:50.813683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:50.813683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:50.813683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:50.813683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:50.813683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:50.813683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:50.813683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:50.813683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:50.813683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:50.813683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:50.813683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:50.813683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:50.813683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:50.813683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:50.813683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:50.814479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:50.814479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:50.814479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:50.814479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:50.814479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:50.814479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:50.814479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:50.814479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:50.814479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:50.814479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:50.814479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:35:50.814479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:50.818945 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 275/544] 10.222.0.115 () {56 vars in 1107 bytes} [Tue Aug 2 10:35:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:50.820369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792f271-39d3-424b-a9f9-5ca4d884c65c None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:50.867541 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66008c-7161-437e-bebd-db3e10c21d39 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:50.867748 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66008c-7161-437e-bebd-db3e10c21d39 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:50.868030 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd66008c-7161-437e-bebd-db3e10c21d39 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:50.869662 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 276/545] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:50 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:50.875065 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ced61a-7ce8-402a-95a0-785e6b960b8a None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:50.875477 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ced61a-7ce8-402a-95a0-785e6b960b8a None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:50.875758 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ced61a-7ce8-402a-95a0-785e6b960b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:50.905198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-95ced61a-7ce8-402a-95a0-785e6b960b8a None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:50.906955 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-95ced61a-7ce8-402a-95a0-785e6b960b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:50.916489 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:50.967825 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792f271-39d3-424b-a9f9-5ca4d884c65c None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:51.018173 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca49d0a-49a2-427b-ad0f-23cb1c3fb655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:51.018809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca49d0a-49a2-427b-ad0f-23cb1c3fb655 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:51.020589 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca49d0a-49a2-427b-ad0f-23cb1c3fb655 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:51.020880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca49d0a-49a2-427b-ad0f-23cb1c3fb655 None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:51.021615 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca49d0a-49a2-427b-ad0f-23cb1c3fb655 None admin] RBAC: Adding query filter params (name=service) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:51.021919 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca49d0a-49a2-427b-ad0f-23cb1c3fb655 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:51.023324 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca49d0a-49a2-427b-ad0f-23cb1c3fb655 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:51.023930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:51.044610 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 270/546] 10.222.0.115 () {56 vars in 1108 bytes} [Tue Aug 2 10:35:50 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:51.051184 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca49d0a-49a2-427b-ad0f-23cb1c3fb655 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:51.067879 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 277/547] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:50 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:35:51.091532 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ced61a-7ce8-402a-95a0-785e6b960b8a None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:51.182532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca49d0a-49a2-427b-ad0f-23cb1c3fb655 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:51.222562 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e961877-ebcd-4aca-b890-d368961f1381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:51.223247 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e961877-ebcd-4aca-b890-d368961f1381 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:51.223529 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e961877-ebcd-4aca-b890-d368961f1381 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:51.223804 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e961877-ebcd-4aca-b890-d368961f1381 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:51.227351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:51.248272 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e961877-ebcd-4aca-b890-d368961f1381 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:51.250391 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e961877-ebcd-4aca-b890-d368961f1381 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:51.277682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-4e961877-ebcd-4aca-b890-d368961f1381 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:51.277682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:51.277682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:51.277682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:51.277682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:51.277682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:51.277682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:51.277682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:51.277682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:51.277682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:51.277682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:51.277682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:51.277682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:35:51.277682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:35:51.277682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:35:51.278609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:35:51.278609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:51.278609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:51.278609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:51.278609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:51.278609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:51.278609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:51.278609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:51.278609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:51.278609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:51.278609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:51.278609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:51.278609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:51.278609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:51.278609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:51.278609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:51.278609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:51.279984 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:35:51.279984 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:35:51.279984 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:51.279984 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:51.280209 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ced61a-7ce8-402a-95a0-785e6b960b8a None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:51.285957 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 271/548] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:35:51.286916 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e961877-ebcd-4aca-b890-d368961f1381 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:51.318301 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59eb3c7-5d11-4c25-8195-7e4cd26349cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:51.319403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59eb3c7-5d11-4c25-8195-7e4cd26349cc None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:51.319677 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59eb3c7-5d11-4c25-8195-7e4cd26349cc None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:51.319952 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59eb3c7-5d11-4c25-8195-7e4cd26349cc None admin] PATH_INFO: `/v3/roles/service` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:51.334076 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59eb3c7-5d11-4c25-8195-7e4cd26349cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:51.339620 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-e59eb3c7-5d11-4c25-8195-7e4cd26349cc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:35:51.339620 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:51.339620 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:51.339620 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:51.339620 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:51.339620 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:51.339620 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:51.339620 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:51.339620 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:51.339620 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:51.339620 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:51.339620 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:51.339620 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:51.339620 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:51.339620 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:51.339620 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:51.341786 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:51.341786 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:51.341786 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:51.341786 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:51.341786 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:51.341786 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:51.341786 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:51.341786 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:51.341786 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:51.341786 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:51.341786 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:51.341786 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:51.341786 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:51.341786 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:51.341786 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:51.341786 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:51.343992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:51.343992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:51.343992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:51.343992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:51.343992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:51.343992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:51.343992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:51.343992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:51.343992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:51.343992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:51.343992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:35:51.343992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:51.343992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 278/549] 10.222.0.115 () {56 vars in 1107 bytes} [Tue Aug 2 10:35:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:35:51.348396 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59eb3c7-5d11-4c25-8195-7e4cd26349cc None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:51.446499 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:51.480747 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e961877-ebcd-4aca-b890-d368961f1381 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:51.521948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38cc8d-1539-43bc-92d6-da9da6981540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:51.522570 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38cc8d-1539-43bc-92d6-da9da6981540 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:51.522853 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38cc8d-1539-43bc-92d6-da9da6981540 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:51.523185 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38cc8d-1539-43bc-92d6-da9da6981540 None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:51.523883 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38cc8d-1539-43bc-92d6-da9da6981540 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:51.524188 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38cc8d-1539-43bc-92d6-da9da6981540 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:51.525569 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab38cc8d-1539-43bc-92d6-da9da6981540 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:51.540991 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 272/550] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:35:51 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:35:51.546768 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38cc8d-1539-43bc-92d6-da9da6981540 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:51.592095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59eb3c7-5d11-4c25-8195-7e4cd26349cc None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:51.608550 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47cd1a-afb0-4727-bd28-8cb8552d52d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:51.609185 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47cd1a-afb0-4727-bd28-8cb8552d52d4 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:51.609454 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47cd1a-afb0-4727-bd28-8cb8552d52d4 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:51.609707 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc47cd1a-afb0-4727-bd28-8cb8552d52d4 None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:51.610420 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc47cd1a-afb0-4727-bd28-8cb8552d52d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:51.610722 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc47cd1a-afb0-4727-bd28-8cb8552d52d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:51.612160 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc47cd1a-afb0-4727-bd28-8cb8552d52d4 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:51.620798 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 279/551] 10.222.0.115 () {56 vars in 1108 bytes} [Tue Aug 2 10:35:51 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:51.626084 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47cd1a-afb0-4727-bd28-8cb8552d52d4 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:51.689077 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:51.778914 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38cc8d-1539-43bc-92d6-da9da6981540 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:51.829214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc47cd1a-afb0-4727-bd28-8cb8552d52d4 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:51.832613 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:51.855705 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24155df3-e169-4e15-adc3-e9770cb206f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:51.857044 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24155df3-e169-4e15-adc3-e9770cb206f7 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:51.857501 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24155df3-e169-4e15-adc3-e9770cb206f7 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:51.857923 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24155df3-e169-4e15-adc3-e9770cb206f7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:51.877031 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24155df3-e169-4e15-adc3-e9770cb206f7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:51.878787 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24155df3-e169-4e15-adc3-e9770cb206f7 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:51.893339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-24155df3-e169-4e15-adc3-e9770cb206f7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:35:51.893339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:51.893339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:51.893339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:51.893339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:51.893339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:51.893339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:51.893339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:51.893339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:51.893339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:51.893339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:51.893339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:51.893339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:35:51.893339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:35:51.893339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:35:51.893339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:35:51.894357 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:51.894357 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:51.894357 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:35:51.894357 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:35:51.894357 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:35:51.894357 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:35:51.894357 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:51.894357 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:51.894357 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:51.894357 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:51.894357 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:51.894357 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:51.894357 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:51.894357 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:51.894357 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:51.894357 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:51.894357 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:51.894357 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:51.895317 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:51.895317 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:51.895317 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:35:51.895317 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:35:51.895317 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:35:51.895317 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:35:51.895317 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:35:51.895317 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:35:51.895317 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:35:51.895317 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:51.898032 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 273/552] 10.222.0.115 () {56 vars in 1098 bytes} [Tue Aug 2 10:35:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:51.908260 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24155df3-e169-4e15-adc3-e9770cb206f7 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:51.915833 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34291c8-67f4-4812-ad12-a76e7a6fe118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:51.916420 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34291c8-67f4-4812-ad12-a76e7a6fe118 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:51.916698 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34291c8-67f4-4812-ad12-a76e7a6fe118 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:51.916966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34291c8-67f4-4812-ad12-a76e7a6fe118 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:51.939777 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34291c8-67f4-4812-ad12-a76e7a6fe118 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:51.941304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34291c8-67f4-4812-ad12-a76e7a6fe118 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:51.966865 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-a34291c8-67f4-4812-ad12-a76e7a6fe118 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:51.966865 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:51.966865 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:51.966865 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:51.966865 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:51.966865 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:51.966865 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:51.966865 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:51.966865 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:51.966865 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:51.966865 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:51.966865 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:51.966865 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:35:51.966865 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:35:51.966865 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:35:51.968562 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:35:51.968562 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:51.968562 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:51.968562 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:51.968562 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:51.968562 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:51.968562 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:51.968562 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:51.968562 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:51.968562 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:51.968562 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:51.968562 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:51.968562 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:51.968562 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:51.968562 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:51.968562 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:51.968562 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:51.969670 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:35:51.969670 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:35:51.969670 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:51.969670 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:51.969891 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 280/553] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:35:51.981836 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34291c8-67f4-4812-ad12-a76e7a6fe118 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:52.100238 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24155df3-e169-4e15-adc3-e9770cb206f7 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:52.101865 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:52.161699 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34291c8-67f4-4812-ad12-a76e7a6fe118 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:52.164308 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c385ec-7654-45ed-93cb-a3ec6bf1f9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:52.164984 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c385ec-7654-45ed-93cb-a3ec6bf1f9c3 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:52.165265 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c385ec-7654-45ed-93cb-a3ec6bf1f9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:52.165542 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c385ec-7654-45ed-93cb-a3ec6bf1f9c3 None admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:52.166328 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c385ec-7654-45ed-93cb-a3ec6bf1f9c3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:52.166631 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c385ec-7654-45ed-93cb-a3ec6bf1f9c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:52.168332 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c385ec-7654-45ed-93cb-a3ec6bf1f9c3 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:52.200412 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 274/554] 10.222.0.115 () {56 vars in 1138 bytes} [Tue Aug 2 10:35:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:52.208107 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c385ec-7654-45ed-93cb-a3ec6bf1f9c3 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:52.237377 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eff23a4-6668-4619-9fa7-2de4866d973e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:52.238417 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff23a4-6668-4619-9fa7-2de4866d973e None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:52.238855 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff23a4-6668-4619-9fa7-2de4866d973e None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:52.239214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eff23a4-6668-4619-9fa7-2de4866d973e None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:52.239929 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eff23a4-6668-4619-9fa7-2de4866d973e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:52.241162 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eff23a4-6668-4619-9fa7-2de4866d973e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:52.242580 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eff23a4-6668-4619-9fa7-2de4866d973e None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:52.271103 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 281/555] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:35:51 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:52.277719 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eff23a4-6668-4619-9fa7-2de4866d973e None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:52.301078 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:52.372456 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c385ec-7654-45ed-93cb-a3ec6bf1f9c3 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:52.414591 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f068859-29b2-4a55-861e-4cee11fb5f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:52.415637 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f068859-29b2-4a55-861e-4cee11fb5f3f None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:52.416127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f068859-29b2-4a55-861e-4cee11fb5f3f None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:52.416258 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eff23a4-6668-4619-9fa7-2de4866d973e None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:52.416731 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f068859-29b2-4a55-861e-4cee11fb5f3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:52.430857 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f068859-29b2-4a55-861e-4cee11fb5f3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:52.432993 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f068859-29b2-4a55-861e-4cee11fb5f3f None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:52.454801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-3f068859-29b2-4a55-861e-4cee11fb5f3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:52.454801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:52.454801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:52.454801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:52.454801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:52.454801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:52.454801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:52.454801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:52.454801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:52.454801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:52.454801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:52.454801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:52.454801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:35:52.454801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:35:52.454801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:35:52.456063 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:35:52.456063 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:52.456063 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:52.456063 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:52.456063 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:52.456063 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:52.456063 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:52.456063 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:52.456063 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:52.456063 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:52.456063 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:52.456063 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:52.456063 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:52.456063 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:52.456063 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:52.456063 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:52.456063 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:52.457408 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:35:52.457408 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:35:52.457408 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:52.457408 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:52.460478 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 275/556] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:52.461986 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa18e82-f864-40be-8e0a-634e6d7a7990 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:52.462742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa18e82-f864-40be-8e0a-634e6d7a7990 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:52.464230 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa18e82-f864-40be-8e0a-634e6d7a7990 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:52.468597 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 276/557] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:52.469732 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa18e82-f864-40be-8e0a-634e6d7a7990 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:52.470379 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fb7a84-aa69-46c8-835e-271b38622c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:52.471064 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb7a84-aa69-46c8-835e-271b38622c28 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:52.471358 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb7a84-aa69-46c8-835e-271b38622c28 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:52.471704 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fb7a84-aa69-46c8-835e-271b38622c28 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:52.482285 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fb7a84-aa69-46c8-835e-271b38622c28 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:52.485033 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fb7a84-aa69-46c8-835e-271b38622c28 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:52.495667 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:52.503004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-02fb7a84-aa69-46c8-835e-271b38622c28 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:35:52.503004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:52.503004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:52.503004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:52.503004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:52.503004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:52.503004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:52.503004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:52.503004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:52.503004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:52.503004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:52.503004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:52.503004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:35:52.503004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:35:52.503004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:35:52.503004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:35:52.504873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:52.504873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:52.504873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:35:52.504873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:35:52.504873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:35:52.504873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:35:52.504873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:52.504873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:52.504873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:52.504873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:52.504873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:52.504873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:52.504873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:52.504873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:52.504873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:52.504873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:52.504873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:52.504873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:52.507651 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:52.507651 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:52.507651 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:35:52.507651 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:35:52.507651 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:35:52.507651 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:35:52.507651 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:35:52.507651 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:35:52.507651 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:35:52.507651 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:52.507651 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 282/558] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:35:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:35:52.507651 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0aa4a-b6fd-4a4b-946d-cebc8c38c0cb None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:52.507651 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0aa4a-b6fd-4a4b-946d-cebc8c38c0cb None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:52.511551 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb0aa4a-b6fd-4a4b-946d-cebc8c38c0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:52.532308 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-1eb0aa4a-b6fd-4a4b-946d-cebc8c38c0cb None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:52.534429 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-1eb0aa4a-b6fd-4a4b-946d-cebc8c38c0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:52.614496 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa18e82-f864-40be-8e0a-634e6d7a7990 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:52.622129 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 283/559] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:52 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:52.623435 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb0aa4a-b6fd-4a4b-946d-cebc8c38c0cb None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:52.632276 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:52.681171 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da593806-e3e0-427d-a80f-f3ba4e52a2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:52.681985 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da593806-e3e0-427d-a80f-f3ba4e52a2cf None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:52.682325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da593806-e3e0-427d-a80f-f3ba4e52a2cf None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:52.682668 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da593806-e3e0-427d-a80f-f3ba4e52a2cf None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:52.683596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da593806-e3e0-427d-a80f-f3ba4e52a2cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:52.683974 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da593806-e3e0-427d-a80f-f3ba4e52a2cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:52.685441 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da593806-e3e0-427d-a80f-f3ba4e52a2cf None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:52.702452 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 277/560] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:35:52 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:52.703965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da593806-e3e0-427d-a80f-f3ba4e52a2cf None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:52.774491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb0aa4a-b6fd-4a4b-946d-cebc8c38c0cb None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:52.804993 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:52.822028 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d2da0-3043-4094-be40-3343f126d992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:52.822832 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d2da0-3043-4094-be40-3343f126d992 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:52.823202 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d2da0-3043-4094-be40-3343f126d992 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:52.823584 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d2da0-3043-4094-be40-3343f126d992 None admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:52.824455 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85d2da0-3043-4094-be40-3343f126d992 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:52.824829 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85d2da0-3043-4094-be40-3343f126d992 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:52.826616 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85d2da0-3043-4094-be40-3343f126d992 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:52.841638 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da593806-e3e0-427d-a80f-f3ba4e52a2cf None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:52.848368 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 284/561] 10.222.0.115 () {56 vars in 1140 bytes} [Tue Aug 2 10:35:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:52.853589 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d2da0-3043-4094-be40-3343f126d992 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:52.880332 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb6001d-f427-473b-a75e-3d819d5aeaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:52.880937 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6001d-f427-473b-a75e-3d819d5aeaa3 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:52.881172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6001d-f427-473b-a75e-3d819d5aeaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:52.881516 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb6001d-f427-473b-a75e-3d819d5aeaa3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:52.893159 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb6001d-f427-473b-a75e-3d819d5aeaa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:52.898599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-9bb6001d-f427-473b-a75e-3d819d5aeaa3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:35:52.898599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:52.898599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:52.898599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:52.898599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:52.898599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:52.898599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:52.898599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:52.898599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:52.898599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:52.898599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:52.898599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:52.898599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:52.898599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:52.898599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:52.898599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:52.899723 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:52.899723 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:52.899723 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:52.899723 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:52.899723 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:52.899723 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:52.899723 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:52.899723 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:52.899723 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:52.899723 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:52.899723 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:52.899723 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:52.899723 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:52.899723 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:52.899723 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:52.899723 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:52.901532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:52.901532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:52.901532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:52.901532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:52.901532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:52.901532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:52.901532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:52.901532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:52.901532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:52.901532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:52.901532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:35:52.901532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:52.901532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 278/562] 10.222.0.115 () {56 vars in 1107 bytes} [Tue Aug 2 10:35:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:52.901532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb6001d-f427-473b-a75e-3d819d5aeaa3 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:52.925806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:52.971628 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d2da0-3043-4094-be40-3343f126d992 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:53.010134 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13fe6b0-5987-4d35-9033-08a3cfe1a03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:53.010808 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13fe6b0-5987-4d35-9033-08a3cfe1a03e None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:53.011120 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13fe6b0-5987-4d35-9033-08a3cfe1a03e None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:53.011389 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13fe6b0-5987-4d35-9033-08a3cfe1a03e None admin] PATH_INFO: `/v3/projects/service` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:53.014937 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb6001d-f427-473b-a75e-3d819d5aeaa3 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:53.018502 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13fe6b0-5987-4d35-9033-08a3cfe1a03e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:53.020029 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13fe6b0-5987-4d35-9033-08a3cfe1a03e None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:53.032208 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-f13fe6b0-5987-4d35-9033-08a3cfe1a03e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:35:53.032208 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:53.032208 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:53.032208 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:53.032208 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:53.032208 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:53.032208 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:53.032208 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:53.032208 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:53.032208 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:53.032208 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:53.032208 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:53.032208 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:35:53.032208 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:35:53.032208 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:35:53.034210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:35:53.034210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:53.034210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:53.034210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:53.034210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:53.034210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:53.034210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:53.034210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:53.034210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:53.034210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:53.034210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:53.034210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:53.034210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:53.034210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:53.034210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:53.034210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:53.034210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:53.035961 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:35:53.035961 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:35:53.035961 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:35:53.035961 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:35:53.035961 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:35:53.035961 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:35:53.035961 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:35:53.035961 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:53.035961 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 285/563] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:35:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:53.035961 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ef0eb-72b9-4b50-b67a-514ea8a2b979 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:53.035961 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ef0eb-72b9-4b50-b67a-514ea8a2b979 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:53.037105 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79ef0eb-72b9-4b50-b67a-514ea8a2b979 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:53.037105 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 286/564] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:53 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:53.037392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ef0eb-72b9-4b50-b67a-514ea8a2b979 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:53.052770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:53.058306 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f043208-b646-4ab5-a99f-fe56e663f7ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:53.058924 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f043208-b646-4ab5-a99f-fe56e663f7ef None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:53.059235 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f043208-b646-4ab5-a99f-fe56e663f7ef None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:53.059510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f043208-b646-4ab5-a99f-fe56e663f7ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:53.132292 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f043208-b646-4ab5-a99f-fe56e663f7ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:53.133647 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f043208-b646-4ab5-a99f-fe56e663f7ef None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:53.150467 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79ef0eb-72b9-4b50-b67a-514ea8a2b979 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:53.174414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:53.187408 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33040147-2029-4d49-b04f-f4ed51dbcba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:53.188531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33040147-2029-4d49-b04f-f4ed51dbcba0 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:53.188531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33040147-2029-4d49-b04f-f4ed51dbcba0 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:53.188531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33040147-2029-4d49-b04f-f4ed51dbcba0 None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:53.194757 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33040147-2029-4d49-b04f-f4ed51dbcba0 None admin] RBAC: Adding query filter params (name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:53.194757 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33040147-2029-4d49-b04f-f4ed51dbcba0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:53.194757 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33040147-2029-4d49-b04f-f4ed51dbcba0 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:53.199983 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 287/565] 10.222.0.115 () {56 vars in 1108 bytes} [Tue Aug 2 10:35:53 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:53.201645 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d7adb-ae09-45b9-893c-a2a97979608d None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:53.202057 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d7adb-ae09-45b9-893c-a2a97979608d None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:53.202572 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d7adb-ae09-45b9-893c-a2a97979608d None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:53.205053 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 288/566] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:53 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:53.206725 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d7adb-ae09-45b9-893c-a2a97979608d None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:53.242483 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 279/567] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:35:52 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:35:53.243933 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09665cb-45fb-4d30-9a5f-4ff109ef8386 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:53.244215 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09665cb-45fb-4d30-9a5f-4ff109ef8386 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:53.244484 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09665cb-45fb-4d30-9a5f-4ff109ef8386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:53.263003 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-d09665cb-45fb-4d30-9a5f-4ff109ef8386 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:53.264291 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-d09665cb-45fb-4d30-9a5f-4ff109ef8386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:53.291206 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:53.319816 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d7adb-ae09-45b9-893c-a2a97979608d None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:53.332569 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 280/568] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:53 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:53.333401 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09665cb-45fb-4d30-9a5f-4ff109ef8386 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:53.358304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a7e9f9-8985-4765-8046-b1bd61c94b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:53.359827 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7e9f9-8985-4765-8046-b1bd61c94b93 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:53.360106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7e9f9-8985-4765-8046-b1bd61c94b93 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:53.360383 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7e9f9-8985-4765-8046-b1bd61c94b93 None admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:53.361135 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a7e9f9-8985-4765-8046-b1bd61c94b93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:53.361439 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a7e9f9-8985-4765-8046-b1bd61c94b93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:53.362877 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a7e9f9-8985-4765-8046-b1bd61c94b93 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:53.376279 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 289/569] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:35:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:53.377542 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0c342-57b5-42ad-8cb6-fde8a71f682c None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:53.377833 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0c342-57b5-42ad-8cb6-fde8a71f682c None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:53.378095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a0c342-57b5-42ad-8cb6-fde8a71f682c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:53.397000 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-f7a0c342-57b5-42ad-8cb6-fde8a71f682c None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:53.398182 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-f7a0c342-57b5-42ad-8cb6-fde8a71f682c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:53.448197 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09665cb-45fb-4d30-9a5f-4ff109ef8386 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:53.448409 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:53.477791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 290/570] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:53 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:53.478356 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0c342-57b5-42ad-8cb6-fde8a71f682c None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:53.488944 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9807fc-c659-4b82-b040-a52efed162ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:53.489913 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9807fc-c659-4b82-b040-a52efed162ed None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:53.489913 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9807fc-c659-4b82-b040-a52efed162ed None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:53.490507 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9807fc-c659-4b82-b040-a52efed162ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:53.504021 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9807fc-c659-4b82-b040-a52efed162ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:53.505478 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9807fc-c659-4b82-b040-a52efed162ed None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:53.521499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-cf9807fc-c659-4b82-b040-a52efed162ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:53.521499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:53.521499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:53.521499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:53.521499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:53.521499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:53.521499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:53.521499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:53.521499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:53.521499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:53.521499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:53.521499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:53.521499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:35:53.521499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:35:53.521499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:35:53.522672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:35:53.522672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:53.522672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:53.522672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:53.522672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:53.522672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:53.522672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:53.522672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:53.522672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:53.522672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:53.522672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:53.522672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:53.522672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:53.522672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:53.522672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:53.522672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:53.522672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:53.523658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:35:53.523658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:35:53.523658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:53.523658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:53.523658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 281/571] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:53.523658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9807fc-c659-4b82-b040-a52efed162ed None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:53.558359 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:53.617186 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a0c342-57b5-42ad-8cb6-fde8a71f682c None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:53.651280 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9807fc-c659-4b82-b040-a52efed162ed None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:53.656663 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3ed37-c8b5-411d-8cf4-4ceec45d06d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:53.657309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3ed37-c8b5-411d-8cf4-4ceec45d06d9 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:53.657586 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3ed37-c8b5-411d-8cf4-4ceec45d06d9 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:53.657861 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae3ed37-c8b5-411d-8cf4-4ceec45d06d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:53.668235 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae3ed37-c8b5-411d-8cf4-4ceec45d06d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:53.669669 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae3ed37-c8b5-411d-8cf4-4ceec45d06d9 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:53.684825 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917cd228-6948-4f14-9fd7-49d00cf7223b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:53.685460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cd228-6948-4f14-9fd7-49d00cf7223b None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:53.685676 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-bae3ed37-c8b5-411d-8cf4-4ceec45d06d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:53.685676 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:53.685676 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:53.685676 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:53.685676 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:53.685676 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:53.685676 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:53.685676 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:53.685676 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:53.685676 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:53.685676 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:53.685676 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:53.685676 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:35:53.685676 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:35:53.685676 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:35:53.687275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:35:53.687275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:53.687275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:53.687275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:53.687275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:53.687275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:53.687275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:53.687275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:53.687275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:53.687275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:53.687275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:53.687275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:53.687275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:53.687275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:53.687275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:53.687275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:53.687275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:53.688171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:35:53.688171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:35:53.688171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:53.688171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:53.688476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cd228-6948-4f14-9fd7-49d00cf7223b None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:53.688476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917cd228-6948-4f14-9fd7-49d00cf7223b None admin] PATH_INFO: `/v3/roles/service` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:53.688609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 291/572] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:53.688878 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3ed37-c8b5-411d-8cf4-4ceec45d06d9 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:53.698595 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917cd228-6948-4f14-9fd7-49d00cf7223b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:53.704931 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-917cd228-6948-4f14-9fd7-49d00cf7223b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:35:53.704931 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:53.704931 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:53.704931 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:53.704931 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:53.704931 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:53.704931 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:53.704931 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:53.704931 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:53.704931 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:53.704931 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:53.704931 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:53.704931 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:53.704931 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:53.704931 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:53.704931 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:53.706169 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:53.706169 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:53.706169 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:53.706169 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:53.706169 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:53.706169 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:53.706169 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:53.706169 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:53.706169 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:53.706169 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:53.706169 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:53.706169 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:53.706169 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:53.706169 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:53.706169 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:53.706169 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:53.707029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:53.707029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:53.707029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:53.707029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:53.707029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:53.707029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:53.707029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:53.707029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:53.707029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:53.707029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:53.707029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:35:53.707029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:53.707029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 282/573] 10.222.0.115 () {56 vars in 1107 bytes} [Tue Aug 2 10:35:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:53.707767 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917cd228-6948-4f14-9fd7-49d00cf7223b None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:53.730370 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:53.799638 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:53.824261 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae3ed37-c8b5-411d-8cf4-4ceec45d06d9 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:53.826527 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917cd228-6948-4f14-9fd7-49d00cf7223b None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:53.862782 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a876df9-f583-4d2f-8a26-8cd8396033eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:53.863488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a876df9-f583-4d2f-8a26-8cd8396033eb None admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:53.863770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a876df9-f583-4d2f-8a26-8cd8396033eb None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:53.864049 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a876df9-f583-4d2f-8a26-8cd8396033eb None admin] PATH_INFO: `/v3/projects/c1eae881e5714bf4bd223809e1a2d920/users/d0478bd540f446ec810e2091c46ac49c/roles/0f1c61dab22c4d97ae784468058036c7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:53.866730 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f88d2-9031-4c39-98f7-e771c30f42fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:53.867407 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f88d2-9031-4c39-98f7-e771c30f42fc None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:53.867688 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f88d2-9031-4c39-98f7-e771c30f42fc None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:53.867964 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0f88d2-9031-4c39-98f7-e771c30f42fc None admin] PATH_INFO: `/v3/roles/service` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:53.878960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0f88d2-9031-4c39-98f7-e771c30f42fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:53.880231 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a876df9-f583-4d2f-8a26-8cd8396033eb None admin] RBAC: Authorizing `identity:create_grant(project_id=c1eae881e5714bf4bd223809e1a2d920, user_id=d0478bd540f446ec810e2091c46ac49c, role_id=0f1c61dab22c4d97ae784468058036c7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:53.881690 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a876df9-f583-4d2f-8a26-8cd8396033eb None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:53.884375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-ac0f88d2-9031-4c39-98f7-e771c30f42fc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:35:53.884375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:53.884375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:53.884375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:53.884375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:53.884375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:53.884375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:53.884375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:53.884375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:53.884375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:53.884375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:53.884375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:53.884375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:53.884375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:53.884375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:53.884375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:53.885697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:53.885697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:53.885697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:53.885697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:53.885697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:53.885697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:53.885697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:53.885697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:53.885697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:53.885697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:53.885697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:53.885697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:53.885697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:53.885697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:53.885697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:53.885697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:53.886536 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:53.886536 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:53.886536 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:53.886536 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:53.886536 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:53.886536 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:53.886536 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:53.886536 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:53.886536 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:53.886536 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:53.886536 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:35:53.886536 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:53.886536 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 283/574] 10.222.0.115 () {56 vars in 1107 bytes} [Tue Aug 2 10:35:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:53.887274 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f88d2-9031-4c39-98f7-e771c30f42fc None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:53.953713 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 292/575] 10.222.0.115 () {58 vars in 1444 bytes} [Tue Aug 2 10:35:53 2022] PUT /identity/v3/projects/c1eae881e5714bf4bd223809e1a2d920/users/d0478bd540f446ec810e2091c46ac49c/roles/0f1c61dab22c4d97ae784468058036c7 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:35:53.954693 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a876df9-f583-4d2f-8a26-8cd8396033eb None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:54.007433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0f88d2-9031-4c39-98f7-e771c30f42fc None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:54.056936 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d7193-c981-408a-941b-855e8ae49b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:54.057574 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d7193-c981-408a-941b-855e8ae49b05 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:54.057853 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d7193-c981-408a-941b-855e8ae49b05 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:54.058134 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d7193-c981-408a-941b-855e8ae49b05 None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:54.058846 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d7193-c981-408a-941b-855e8ae49b05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:54.059176 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d7193-c981-408a-941b-855e8ae49b05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:54.060698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502d7193-c981-408a-941b-855e8ae49b05 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:54.063417 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:54.066399 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:54.082154 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 284/576] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:35:53 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:54.082154 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d7193-c981-408a-941b-855e8ae49b05 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:54.097820 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a876df9-f583-4d2f-8a26-8cd8396033eb None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:54.143916 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb622f1-608e-48f4-b0ce-b38597555f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:54.144565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb622f1-608e-48f4-b0ce-b38597555f79 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:54.144860 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb622f1-608e-48f4-b0ce-b38597555f79 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:54.145132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb622f1-608e-48f4-b0ce-b38597555f79 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:54.145920 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb622f1-608e-48f4-b0ce-b38597555f79 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:54.146231 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb622f1-608e-48f4-b0ce-b38597555f79 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:54.147683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb622f1-608e-48f4-b0ce-b38597555f79 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:54.165663 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 293/577] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:35:53 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:54.166606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb622f1-608e-48f4-b0ce-b38597555f79 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:54.225017 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d7193-c981-408a-941b-855e8ae49b05 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:54.278673 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af74e6a-2286-4e6d-8d0d-69dd20eb29aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:54.279368 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af74e6a-2286-4e6d-8d0d-69dd20eb29aa None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:54.279652 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af74e6a-2286-4e6d-8d0d-69dd20eb29aa None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:54.279923 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af74e6a-2286-4e6d-8d0d-69dd20eb29aa None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:54.280680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af74e6a-2286-4e6d-8d0d-69dd20eb29aa None admin] RBAC: Adding query filter params (name=service) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:54.281026 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af74e6a-2286-4e6d-8d0d-69dd20eb29aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:54.282641 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af74e6a-2286-4e6d-8d0d-69dd20eb29aa None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:54.291088 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb622f1-608e-48f4-b0ce-b38597555f79 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:54.291654 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 285/578] 10.222.0.115 () {56 vars in 1108 bytes} [Tue Aug 2 10:35:54 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:54.292482 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af74e6a-2286-4e6d-8d0d-69dd20eb29aa None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:54.336835 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a6692-26fe-4641-aebc-511b501457cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:54.337874 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a6692-26fe-4641-aebc-511b501457cb None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:54.338424 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a6692-26fe-4641-aebc-511b501457cb None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:54.338699 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a6692-26fe-4641-aebc-511b501457cb None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:54.339457 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a6692-26fe-4641-aebc-511b501457cb None admin] RBAC: Adding query filter params (name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:54.339759 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a6692-26fe-4641-aebc-511b501457cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:54.341132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a6692-26fe-4641-aebc-511b501457cb None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:54.344214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:54.357245 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 294/579] 10.222.0.115 () {56 vars in 1108 bytes} [Tue Aug 2 10:35:54 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:54.358189 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a6692-26fe-4641-aebc-511b501457cb None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:54.364589 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:54.414463 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af74e6a-2286-4e6d-8d0d-69dd20eb29aa None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:54.460295 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d40f322-05cc-4c82-8a62-aa41a31bab35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:54.461290 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40f322-05cc-4c82-8a62-aa41a31bab35 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:54.461719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40f322-05cc-4c82-8a62-aa41a31bab35 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:54.462173 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d40f322-05cc-4c82-8a62-aa41a31bab35 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:54.472773 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d40f322-05cc-4c82-8a62-aa41a31bab35 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:54.474264 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d40f322-05cc-4c82-8a62-aa41a31bab35 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:54.482584 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a6692-26fe-4641-aebc-511b501457cb None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:54.487395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-2d40f322-05cc-4c82-8a62-aa41a31bab35 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 10:35:54.487395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:54.487395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:54.487395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:54.487395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:54.487395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:54.487395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:54.487395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:54.487395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:54.487395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:54.487395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:54.487395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:54.487395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:35:54.487395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:35:54.487395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:35:54.487395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:35:54.491742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:54.491742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:54.491742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:35:54.491742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:35:54.491742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:35:54.491742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:35:54.491742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:54.491742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:54.491742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:54.491742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:54.491742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:54.491742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:54.491742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:54.491742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:54.491742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:54.491742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:54.491742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:54.491742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:54.492784 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:54.492784 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:54.492784 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:35:54.492784 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:35:54.492784 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:35:54.492784 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:35:54.492784 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:35:54.492784 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:35:54.492784 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 10:35:54.492784 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:54.493354 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 286/580] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:35:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:54.494867 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d40f322-05cc-4c82-8a62-aa41a31bab35 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:54.525866 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86203ad-c569-4de4-8f0e-dd8f2838ca1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:54.526941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86203ad-c569-4de4-8f0e-dd8f2838ca1a None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:54.527379 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86203ad-c569-4de4-8f0e-dd8f2838ca1a None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:54.527660 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86203ad-c569-4de4-8f0e-dd8f2838ca1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:54.535491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86203ad-c569-4de4-8f0e-dd8f2838ca1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:54.536932 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86203ad-c569-4de4-8f0e-dd8f2838ca1a None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:54.548395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-f86203ad-c569-4de4-8f0e-dd8f2838ca1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:35:54.548395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:54.548395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:54.548395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:54.548395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:54.548395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:54.548395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:54.548395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:54.548395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:54.548395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:54.548395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:54.548395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:54.548395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:35:54.548395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:35:54.548395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:35:54.549509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:35:54.549509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:54.549509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:54.549509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:54.549509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:54.549509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:54.549509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:54.549509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:54.549509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:54.549509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:54.549509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:54.549509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:54.549509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:54.549509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:54.549509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:54.549509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:54.549509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:54.550774 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:35:54.550774 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:35:54.550774 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:35:54.550774 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:35:54.550774 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:35:54.550774 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:35:54.550774 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:35:54.550774 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:54.550774 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 295/581] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:35:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:54.550774 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86203ad-c569-4de4-8f0e-dd8f2838ca1a None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:54.617620 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:54.630263 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d40f322-05cc-4c82-8a62-aa41a31bab35 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:54.652946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:54.662210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86203ad-c569-4de4-8f0e-dd8f2838ca1a None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:54.672998 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20436fd6-882a-4d46-8407-7f804bf97ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:54.673644 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20436fd6-882a-4d46-8407-7f804bf97ef2 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:54.673927 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20436fd6-882a-4d46-8407-7f804bf97ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:54.674204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20436fd6-882a-4d46-8407-7f804bf97ef2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:54.686250 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20436fd6-882a-4d46-8407-7f804bf97ef2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:54.688615 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20436fd6-882a-4d46-8407-7f804bf97ef2 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:54.698293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f64a6-b3bf-4c0a-aed0-716170114fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:54.698970 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f64a6-b3bf-4c0a-aed0-716170114fb7 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:54.699295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f64a6-b3bf-4c0a-aed0-716170114fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:54.699570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2f64a6-b3bf-4c0a-aed0-716170114fb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:54.704532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-20436fd6-882a-4d46-8407-7f804bf97ef2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:54.704532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:54.704532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:54.704532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:54.704532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:54.704532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:54.704532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:54.704532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:54.704532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:54.704532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:54.704532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:54.704532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:54.704532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:35:54.704532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:35:54.704532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:35:54.705698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:35:54.705698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:54.705698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:54.705698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:54.705698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:54.705698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:54.705698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:54.705698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:54.705698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:54.705698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:54.705698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:54.705698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:54.705698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:54.705698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:54.705698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:54.705698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:54.705698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:54.711463 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:35:54.711463 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:35:54.711463 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:54.711463 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:54.711463 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 287/582] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:54.711463 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20436fd6-882a-4d46-8407-7f804bf97ef2 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:54.716221 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2f64a6-b3bf-4c0a-aed0-716170114fb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:54.717872 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2f64a6-b3bf-4c0a-aed0-716170114fb7 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:54.735729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-ab2f64a6-b3bf-4c0a-aed0-716170114fb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:54.735729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:54.735729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:54.735729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:54.735729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:54.735729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:54.735729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:54.735729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:54.735729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:54.735729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:54.735729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:54.735729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:54.735729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:35:54.735729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:35:54.735729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:35:54.736927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:35:54.736927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:54.736927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:54.736927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:54.736927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:54.736927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:54.736927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:54.736927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:54.736927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:54.736927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:54.736927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:54.736927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:54.736927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:54.736927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:54.736927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:54.736927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:54.736927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:54.738544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:35:54.738544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:35:54.738544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:54.738544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:54.738544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 296/583] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:54.739322 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f64a6-b3bf-4c0a-aed0-716170114fb7 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:54.825899 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20436fd6-882a-4d46-8407-7f804bf97ef2 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:54.849374 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2f64a6-b3bf-4c0a-aed0-716170114fb7 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:54.860818 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:54.861176 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6e860-8e26-4f8f-994c-b2a7fe23da24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:54.861785 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6e860-8e26-4f8f-994c-b2a7fe23da24 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:54.862072 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6e860-8e26-4f8f-994c-b2a7fe23da24 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:54.862347 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f6e860-8e26-4f8f-994c-b2a7fe23da24 None admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:54.863178 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6e860-8e26-4f8f-994c-b2a7fe23da24 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:54.863491 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6e860-8e26-4f8f-994c-b2a7fe23da24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:54.864927 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f6e860-8e26-4f8f-994c-b2a7fe23da24 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:54.886181 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df2a3a-7acb-4447-9a4b-44e3e7cae278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:54.886870 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df2a3a-7acb-4447-9a4b-44e3e7cae278 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:54.887181 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df2a3a-7acb-4447-9a4b-44e3e7cae278 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:54.887458 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86df2a3a-7acb-4447-9a4b-44e3e7cae278 None admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:54.888217 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df2a3a-7acb-4447-9a4b-44e3e7cae278 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:54.888526 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df2a3a-7acb-4447-9a4b-44e3e7cae278 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:54.890065 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86df2a3a-7acb-4447-9a4b-44e3e7cae278 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:54.897451 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 288/584] 10.222.0.115 () {56 vars in 1142 bytes} [Tue Aug 2 10:35:54 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:54.898298 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6e860-8e26-4f8f-994c-b2a7fe23da24 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:54.907859 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 297/585] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:35:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:54.909032 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df2a3a-7acb-4447-9a4b-44e3e7cae278 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:54.939214 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:55.032606 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f6e860-8e26-4f8f-994c-b2a7fe23da24 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:55.052950 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86df2a3a-7acb-4447-9a4b-44e3e7cae278 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:55.078483 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2d1ca-d944-4d95-9f72-8450c9646649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:55.079152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2d1ca-d944-4d95-9f72-8450c9646649 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:55.079435 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2d1ca-d944-4d95-9f72-8450c9646649 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:55.079713 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb2d1ca-d944-4d95-9f72-8450c9646649 None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:55.080680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb2d1ca-d944-4d95-9f72-8450c9646649 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:55.080987 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb2d1ca-d944-4d95-9f72-8450c9646649 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:55.082392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb2d1ca-d944-4d95-9f72-8450c9646649 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:55.100664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 289/586] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:35:54 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:55.100664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2d1ca-d944-4d95-9f72-8450c9646649 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:55.100970 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a788e-dcd0-462c-bd20-bde7a84c32d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:55.101634 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a788e-dcd0-462c-bd20-bde7a84c32d4 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:55.101923 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a788e-dcd0-462c-bd20-bde7a84c32d4 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:55.102201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847a788e-dcd0-462c-bd20-bde7a84c32d4 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:55.102935 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a788e-dcd0-462c-bd20-bde7a84c32d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:55.103278 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a788e-dcd0-462c-bd20-bde7a84c32d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:55.104707 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847a788e-dcd0-462c-bd20-bde7a84c32d4 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:55.118475 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 298/587] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:35:54 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:55.119497 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a788e-dcd0-462c-bd20-bde7a84c32d4 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:55.153640 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:55.210534 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:55.227242 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb2d1ca-d944-4d95-9f72-8450c9646649 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:55.230521 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847a788e-dcd0-462c-bd20-bde7a84c32d4 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:55.268331 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c188a-cf3b-4e2a-b3b4-0da0605290d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:55.268832 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164634c-b15a-4d0f-8260-658d1583eedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:55.268983 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c188a-cf3b-4e2a-b3b4-0da0605290d8 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:55.269218 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c188a-cf3b-4e2a-b3b4-0da0605290d8 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:55.269472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164634c-b15a-4d0f-8260-658d1583eedf None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:55.269572 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268c188a-cf3b-4e2a-b3b4-0da0605290d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:55.269753 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164634c-b15a-4d0f-8260-658d1583eedf None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:55.270028 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4164634c-b15a-4d0f-8260-658d1583eedf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:55.270269 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268c188a-cf3b-4e2a-b3b4-0da0605290d8 None admin] RBAC: Adding query filter params (role.id=0f1c61dab22c4d97ae784468058036c7, scope.project.id=c1eae881e5714bf4bd223809e1a2d920, user.id=0daa49b5a35f4a468bb8d0939e7ec20c) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:55.270578 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268c188a-cf3b-4e2a-b3b4-0da0605290d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:55.272048 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268c188a-cf3b-4e2a-b3b4-0da0605290d8 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:55.278055 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 299/588] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:35:55 2022] GET /identity/v3/role_assignments?user.id=0daa49b5a35f4a468bb8d0939e7ec20c&scope.project.id=c1eae881e5714bf4bd223809e1a2d920&role.id=0f1c61dab22c4d97ae784468058036c7 => generated 260 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:55.278873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c188a-cf3b-4e2a-b3b4-0da0605290d8 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:55.292233 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164634c-b15a-4d0f-8260-658d1583eedf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:55.294057 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4164634c-b15a-4d0f-8260-658d1583eedf None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:55.312539 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-4164634c-b15a-4d0f-8260-658d1583eedf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:55.312539 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:55.312539 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:55.312539 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:55.312539 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:55.312539 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:55.312539 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:55.312539 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:55.312539 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:55.312539 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:55.312539 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:55.312539 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:55.312539 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:35:55.312539 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:35:55.312539 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:35:55.313796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:35:55.313796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:55.313796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:55.313796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:55.313796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:55.313796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:55.313796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:55.313796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:55.313796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:55.313796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:55.313796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:55.313796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:55.313796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:55.313796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:55.313796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:55.313796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:55.313796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:55.314953 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:35:55.314953 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:35:55.314953 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:55.314953 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:55.318178 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 290/589] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:55.320626 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164634c-b15a-4d0f-8260-658d1583eedf None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:55.422186 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:55.442091 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268c188a-cf3b-4e2a-b3b4-0da0605290d8 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:55.483095 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4164634c-b15a-4d0f-8260-658d1583eedf None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:55.492106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b1214-568d-4b6c-b499-44163c76093b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:55.492764 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b1214-568d-4b6c-b499-44163c76093b None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:55.493048 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b1214-568d-4b6c-b499-44163c76093b None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:55.493382 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31b1214-568d-4b6c-b499-44163c76093b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:55.503895 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b1214-568d-4b6c-b499-44163c76093b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:55.505655 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31b1214-568d-4b6c-b499-44163c76093b None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:55.520499 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-a31b1214-568d-4b6c-b499-44163c76093b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 10:35:55.520499 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:55.520499 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:55.520499 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:55.520499 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:55.520499 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:55.520499 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:55.520499 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:55.520499 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:55.520499 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:55.520499 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:55.520499 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:55.520499 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:35:55.520499 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:35:55.520499 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:35:55.520499 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:35:55.521612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:55.521612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:55.521612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:35:55.521612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:35:55.521612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:35:55.521612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:35:55.521612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:55.521612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:55.521612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:55.521612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:55.521612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:55.521612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:55.521612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:55.521612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:55.521612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:55.521612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:55.521612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:55.521612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:55.522559 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:55.522559 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:55.522559 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:35:55.522559 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:35:55.522559 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:35:55.522559 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:35:55.522559 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:35:55.522559 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:35:55.522559 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 10:35:55.522559 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:55.526576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 300/590] 10.222.0.115 () {56 vars in 1107 bytes} [Tue Aug 2 10:35:55 2022] GET /identity/v3/users/neutron => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:55.527738 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b1214-568d-4b6c-b499-44163c76093b None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:55.533850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6097fe-7747-4f6e-b8f4-a8c7f12baacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:55.534935 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6097fe-7747-4f6e-b8f4-a8c7f12baacc None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:55.535448 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6097fe-7747-4f6e-b8f4-a8c7f12baacc None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:55.535947 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6097fe-7747-4f6e-b8f4-a8c7f12baacc None admin] PATH_INFO: `/v3/users/glance` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:55.548003 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6097fe-7747-4f6e-b8f4-a8c7f12baacc None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:55.549399 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6097fe-7747-4f6e-b8f4-a8c7f12baacc None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:55.551966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:55.568419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-4b6097fe-7747-4f6e-b8f4-a8c7f12baacc None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 02 10:35:55.568419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:55.568419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:55.568419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:55.568419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:55.568419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:55.568419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:55.568419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:55.568419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:55.568419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:55.568419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:55.568419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:55.568419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:35:55.568419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:35:55.568419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:35:55.568419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:35:55.569460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:55.569460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:55.569460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:35:55.569460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:35:55.569460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:35:55.569460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:35:55.569460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:55.569460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:55.569460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:55.569460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:55.569460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:55.569460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:55.569460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:55.569460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:55.569460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:55.569460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:55.569460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:55.569460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:55.570437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:55.570437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:55.570437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:35:55.570437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:35:55.570437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:35:55.570437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:35:55.570437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:35:55.570437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:35:55.570437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 02 10:35:55.570437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:55.572333 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 291/591] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:35:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:55.583117 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6097fe-7747-4f6e-b8f4-a8c7f12baacc None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:55.661101 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31b1214-568d-4b6c-b499-44163c76093b None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:55.697380 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6097fe-7747-4f6e-b8f4-a8c7f12baacc None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:55.703438 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c270ed5-b0f5-4126-a379-0d1507d15dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:55.704081 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c270ed5-b0f5-4126-a379-0d1507d15dac None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:55.704364 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c270ed5-b0f5-4126-a379-0d1507d15dac None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:55.704641 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c270ed5-b0f5-4126-a379-0d1507d15dac None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:55.705635 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c270ed5-b0f5-4126-a379-0d1507d15dac None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:55.706064 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c270ed5-b0f5-4126-a379-0d1507d15dac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:55.707537 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c270ed5-b0f5-4126-a379-0d1507d15dac None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:55.722307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 301/592] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:35:55 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:55.723285 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c270ed5-b0f5-4126-a379-0d1507d15dac None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:55.737152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49eabe-7857-4348-a06b-3f558dbf953b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:55.737783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49eabe-7857-4348-a06b-3f558dbf953b None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:55.738064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49eabe-7857-4348-a06b-3f558dbf953b None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:55.738346 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49eabe-7857-4348-a06b-3f558dbf953b None admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:55.739181 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49eabe-7857-4348-a06b-3f558dbf953b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:55.739595 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49eabe-7857-4348-a06b-3f558dbf953b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:55.740567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:55.742266 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49eabe-7857-4348-a06b-3f558dbf953b None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:55.766800 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 292/593] 10.222.0.115 () {56 vars in 1144 bytes} [Tue Aug 2 10:35:55 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:55.768162 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49eabe-7857-4348-a06b-3f558dbf953b None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:55.840999 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c270ed5-b0f5-4126-a379-0d1507d15dac None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:55.857026 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:55.895820 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00866c-cd35-408b-877b-157422cdc7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:55.896772 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00866c-cd35-408b-877b-157422cdc7fe None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:55.897072 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00866c-cd35-408b-877b-157422cdc7fe None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:55.897400 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af00866c-cd35-408b-877b-157422cdc7fe None admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:55.898161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af00866c-cd35-408b-877b-157422cdc7fe None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:55.898503 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af00866c-cd35-408b-877b-157422cdc7fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:55.900077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af00866c-cd35-408b-877b-157422cdc7fe None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:55.924872 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 302/594] 10.222.0.115 () {56 vars in 1142 bytes} [Tue Aug 2 10:35:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:55.924872 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00866c-cd35-408b-877b-157422cdc7fe None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:55.932435 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49eabe-7857-4348-a06b-3f558dbf953b None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:55.976075 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861e188-c03a-48d2-97aa-8a3b718738a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:55.977166 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861e188-c03a-48d2-97aa-8a3b718738a9 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:55.977645 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861e188-c03a-48d2-97aa-8a3b718738a9 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:55.978444 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861e188-c03a-48d2-97aa-8a3b718738a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:55.987433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5861e188-c03a-48d2-97aa-8a3b718738a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:55.989388 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5861e188-c03a-48d2-97aa-8a3b718738a9 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:56.004334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-5861e188-c03a-48d2-97aa-8a3b718738a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:35:56.004334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:56.004334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:56.004334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:56.004334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:56.004334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:56.004334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:56.004334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:56.004334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:56.004334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:56.004334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:56.004334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:56.004334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:35:56.004334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:35:56.004334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:35:56.005433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:35:56.005433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:56.005433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:56.005433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:56.005433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:56.005433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:56.005433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:56.005433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:56.005433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:56.005433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:56.005433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:56.005433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:56.005433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:56.005433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:56.005433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:56.005433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:56.005433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:56.006459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:35:56.006459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:35:56.006459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:35:56.006459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:35:56.006459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:35:56.006459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:35:56.006459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:35:56.006459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:56.007306 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:56.009898 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 293/595] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:35:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:56.011167 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861e188-c03a-48d2-97aa-8a3b718738a9 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:56.041906 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af00866c-cd35-408b-877b-157422cdc7fe None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:56.083795 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c632ac0-330e-40b0-953d-d89b1a77999a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:56.084792 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c632ac0-330e-40b0-953d-d89b1a77999a None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:56.085219 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c632ac0-330e-40b0-953d-d89b1a77999a None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:56.085696 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c632ac0-330e-40b0-953d-d89b1a77999a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:56.099713 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c632ac0-330e-40b0-953d-d89b1a77999a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:56.101888 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c632ac0-330e-40b0-953d-d89b1a77999a None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:56.124632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-6c632ac0-330e-40b0-953d-d89b1a77999a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:56.124632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:56.124632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:56.124632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:56.124632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:56.124632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:56.124632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:56.124632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:56.124632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:56.124632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:56.124632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:56.124632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:56.124632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:35:56.124632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:35:56.124632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:35:56.125659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:35:56.125659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:56.125659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:56.125659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:56.125659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:56.125659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:56.125659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:56.125659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:56.125659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:56.125659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:56.125659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:56.125659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:56.125659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:56.125659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:56.125659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:56.125659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:56.125659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:56.126567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:35:56.126567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:35:56.126567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:56.126567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:56.126567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 303/596] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:56.127716 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c632ac0-330e-40b0-953d-d89b1a77999a None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:56.134615 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5861e188-c03a-48d2-97aa-8a3b718738a9 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:56.154219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:56.183563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9683eb7-bb91-418e-bab4-2564ab232374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:56.184655 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9683eb7-bb91-418e-bab4-2564ab232374 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:56.185148 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9683eb7-bb91-418e-bab4-2564ab232374 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:56.185719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9683eb7-bb91-418e-bab4-2564ab232374 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:56.207003 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9683eb7-bb91-418e-bab4-2564ab232374 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:56.212066 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9683eb7-bb91-418e-bab4-2564ab232374 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:56.232075 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-e9683eb7-bb91-418e-bab4-2564ab232374 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:56.232075 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:56.232075 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:56.232075 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:56.232075 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:56.232075 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:56.232075 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:56.232075 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:56.232075 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:56.232075 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:56.232075 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:56.232075 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:56.232075 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:35:56.232075 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:35:56.232075 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:35:56.233223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:35:56.233223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:56.233223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:56.233223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:56.233223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:56.233223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:56.233223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:56.233223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:56.233223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:56.233223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:56.233223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:56.233223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:56.233223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:56.233223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:56.233223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:56.233223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:56.233223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:56.235586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:35:56.235586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:35:56.235586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:56.235586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:56.238982 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 294/597] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:56.240276 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e398bf3-5ee0-43f7-826d-aac16693ced7 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:56.240556 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e398bf3-5ee0-43f7-826d-aac16693ced7 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:56.240831 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e398bf3-5ee0-43f7-826d-aac16693ced7 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:56.242434 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 295/598] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:56.243183 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e398bf3-5ee0-43f7-826d-aac16693ced7 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:56.299800 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c632ac0-330e-40b0-953d-d89b1a77999a None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:56.335069 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:56.349262 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c944dab2-1dc6-4ac1-a95a-c4e0da2aa970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:56.354601 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c944dab2-1dc6-4ac1-a95a-c4e0da2aa970 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:56.354601 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c944dab2-1dc6-4ac1-a95a-c4e0da2aa970 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:56.354601 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c944dab2-1dc6-4ac1-a95a-c4e0da2aa970 None admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:56.354601 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c944dab2-1dc6-4ac1-a95a-c4e0da2aa970 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:56.354601 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c944dab2-1dc6-4ac1-a95a-c4e0da2aa970 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:56.354601 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c944dab2-1dc6-4ac1-a95a-c4e0da2aa970 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:56.377118 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 304/599] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:35:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:56.377118 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c944dab2-1dc6-4ac1-a95a-c4e0da2aa970 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:56.398288 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e398bf3-5ee0-43f7-826d-aac16693ced7 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:56.450268 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e36b3e3-4634-4a60-b438-67894068ddc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:56.451018 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36b3e3-4634-4a60-b438-67894068ddc0 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:56.451331 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36b3e3-4634-4a60-b438-67894068ddc0 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:56.451610 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36b3e3-4634-4a60-b438-67894068ddc0 None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:56.455292 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e36b3e3-4634-4a60-b438-67894068ddc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:56.456147 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e36b3e3-4634-4a60-b438-67894068ddc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:56.459029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e36b3e3-4634-4a60-b438-67894068ddc0 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:56.476339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 296/600] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:35:56 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:56.476339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcdf66e-d3cf-47ff-8487-8efa21b5a5cd None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:56.476339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcdf66e-d3cf-47ff-8487-8efa21b5a5cd None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:56.476339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcdf66e-d3cf-47ff-8487-8efa21b5a5cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:56.496776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-3fcdf66e-d3cf-47ff-8487-8efa21b5a5cd None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:56.497984 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-3fcdf66e-d3cf-47ff-8487-8efa21b5a5cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:56.539025 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:56.544558 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c944dab2-1dc6-4ac1-a95a-c4e0da2aa970 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:56.602249 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954f8ae-23a2-4122-bdc2-310860c3dde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:56.603339 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954f8ae-23a2-4122-bdc2-310860c3dde1 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:56.603939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 297/601] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:56 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:56.604696 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcdf66e-d3cf-47ff-8487-8efa21b5a5cd None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:56.607295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954f8ae-23a2-4122-bdc2-310860c3dde1 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:56.607677 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4954f8ae-23a2-4122-bdc2-310860c3dde1 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:56.609263 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954f8ae-23a2-4122-bdc2-310860c3dde1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:56.609578 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954f8ae-23a2-4122-bdc2-310860c3dde1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:56.610992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4954f8ae-23a2-4122-bdc2-310860c3dde1 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:56.637787 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 305/602] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:35:56 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:56.638769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954f8ae-23a2-4122-bdc2-310860c3dde1 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:56.695179 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:56.744329 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcdf66e-d3cf-47ff-8487-8efa21b5a5cd None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:56.770835 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4954f8ae-23a2-4122-bdc2-310860c3dde1 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:56.796788 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fe856-aa84-4c9f-a963-9ef9fde4b953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:56.797608 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fe856-aa84-4c9f-a963-9ef9fde4b953 None admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:56.797989 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fe856-aa84-4c9f-a963-9ef9fde4b953 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:56.798317 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045fe856-aa84-4c9f-a963-9ef9fde4b953 None admin] PATH_INFO: `/v3/projects/c1eae881e5714bf4bd223809e1a2d920/users/cc9ab3c9075b4a8385ef634dc0b1625e/roles/0f1c61dab22c4d97ae784468058036c7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:56.817669 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fe856-aa84-4c9f-a963-9ef9fde4b953 None admin] RBAC: Authorizing `identity:create_grant(project_id=c1eae881e5714bf4bd223809e1a2d920, user_id=cc9ab3c9075b4a8385ef634dc0b1625e, role_id=0f1c61dab22c4d97ae784468058036c7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:56.819294 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045fe856-aa84-4c9f-a963-9ef9fde4b953 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:56.831683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2af05d-1cdd-4555-b08e-5923faa5bc33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:56.832333 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2af05d-1cdd-4555-b08e-5923faa5bc33 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:56.832613 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2af05d-1cdd-4555-b08e-5923faa5bc33 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:56.832886 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2af05d-1cdd-4555-b08e-5923faa5bc33 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:56.840189 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2af05d-1cdd-4555-b08e-5923faa5bc33 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:56.841634 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2af05d-1cdd-4555-b08e-5923faa5bc33 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:56.862297 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-9c2af05d-1cdd-4555-b08e-5923faa5bc33 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:35:56.862297 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:56.862297 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:56.862297 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:56.862297 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:56.862297 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:56.862297 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:56.862297 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:56.862297 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:56.862297 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:56.862297 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:56.862297 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:56.862297 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:35:56.862297 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:35:56.862297 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:35:56.863852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:35:56.863852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:56.863852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:56.863852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:56.863852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:56.863852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:56.863852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:56.863852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:56.863852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:56.863852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:56.863852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:56.863852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:56.863852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:56.863852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:56.863852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:56.863852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:56.863852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:56.865584 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:35:56.865584 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:35:56.865584 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:35:56.865584 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:35:56.865584 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:35:56.865584 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:35:56.865584 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:35:56.865584 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:56.865584 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 306/603] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:35:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:56.865584 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2af05d-1cdd-4555-b08e-5923faa5bc33 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:56.910504 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 298/604] 10.222.0.115 () {58 vars in 1444 bytes} [Tue Aug 2 10:35:56 2022] PUT /identity/v3/projects/c1eae881e5714bf4bd223809e1a2d920/users/cc9ab3c9075b4a8385ef634dc0b1625e/roles/0f1c61dab22c4d97ae784468058036c7 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:35:56.910504 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fe856-aa84-4c9f-a963-9ef9fde4b953 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:56.925211 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:56.996442 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2af05d-1cdd-4555-b08e-5923faa5bc33 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:57.000725 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:57.045212 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045fe856-aa84-4c9f-a963-9ef9fde4b953 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:57.049739 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b148ed8-36d2-4b69-901c-b28ba8893e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:57.050514 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b148ed8-36d2-4b69-901c-b28ba8893e91 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:57.050790 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b148ed8-36d2-4b69-901c-b28ba8893e91 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:57.051073 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b148ed8-36d2-4b69-901c-b28ba8893e91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:57.059315 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b148ed8-36d2-4b69-901c-b28ba8893e91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:57.060808 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b148ed8-36d2-4b69-901c-b28ba8893e91 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:57.074491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-0b148ed8-36d2-4b69-901c-b28ba8893e91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:35:57.074491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:57.074491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:57.074491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:57.074491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:57.074491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:57.074491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:57.074491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:57.074491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:57.074491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:57.074491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:57.074491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:57.074491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:35:57.074491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:35:57.074491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:35:57.075520 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:35:57.075520 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:57.075520 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:57.075520 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:57.075520 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:57.075520 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:57.075520 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:57.075520 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:57.075520 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:57.075520 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:57.075520 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:57.075520 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:57.075520 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:57.075520 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:57.075520 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:57.075520 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:57.075520 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:57.076412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:35:57.076412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:35:57.076412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:35:57.076412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:35:57.076412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:35:57.076412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:35:57.076412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:35:57.076412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:57.081338 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 307/605] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:35:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:57.082341 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b148ed8-36d2-4b69-901c-b28ba8893e91 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:57.101826 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f34ef-2863-4699-a760-a5fd233d098f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:57.102859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f34ef-2863-4699-a760-a5fd233d098f None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:57.103359 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f34ef-2863-4699-a760-a5fd233d098f None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:57.103783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f34ef-2863-4699-a760-a5fd233d098f None admin] PATH_INFO: `/v3/roles/service` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:57.118063 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19f34ef-2863-4699-a760-a5fd233d098f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:57.123253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-c19f34ef-2863-4699-a760-a5fd233d098f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:35:57.123253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:57.123253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:57.123253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:57.123253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:57.123253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:57.123253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:57.123253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:57.123253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:57.123253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:57.123253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:57.123253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:57.123253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:57.123253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:57.123253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:57.123253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:57.124527 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:57.124527 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:57.124527 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:57.124527 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:57.124527 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:57.124527 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:57.124527 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:57.124527 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:57.124527 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:57.124527 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:57.124527 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:57.124527 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:57.124527 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:57.124527 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:57.124527 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:57.124527 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:57.125474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:57.125474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:57.125474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:57.125474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:57.125474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:57.125474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:57.125474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:57.125474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:57.125474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:57.125474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:57.125474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:35:57.125474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:57.126214 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 299/606] 10.222.0.115 () {56 vars in 1107 bytes} [Tue Aug 2 10:35:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:57.128039 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f34ef-2863-4699-a760-a5fd233d098f None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:57.223295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b148ed8-36d2-4b69-901c-b28ba8893e91 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:57.227022 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:57.268509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b312e3d9-f5b6-4660-a19f-41a9623042ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:57.269180 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312e3d9-f5b6-4660-a19f-41a9623042ec None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:57.269469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312e3d9-f5b6-4660-a19f-41a9623042ec None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:57.269778 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b312e3d9-f5b6-4660-a19f-41a9623042ec None admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:57.270589 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b312e3d9-f5b6-4660-a19f-41a9623042ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:57.270900 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b312e3d9-f5b6-4660-a19f-41a9623042ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:57.272677 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b312e3d9-f5b6-4660-a19f-41a9623042ec None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:57.275781 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f34ef-2863-4699-a760-a5fd233d098f None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:57.291693 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 308/607] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:35:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:57.292678 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b312e3d9-f5b6-4660-a19f-41a9623042ec None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:57.313710 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbfc8ac-ecce-4922-b9b5-55e93b3f2453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:57.314927 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfc8ac-ecce-4922-b9b5-55e93b3f2453 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:57.315452 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfc8ac-ecce-4922-b9b5-55e93b3f2453 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:57.315922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfc8ac-ecce-4922-b9b5-55e93b3f2453 None admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:57.316870 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbfc8ac-ecce-4922-b9b5-55e93b3f2453 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:57.317395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbfc8ac-ecce-4922-b9b5-55e93b3f2453 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:57.319203 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbfc8ac-ecce-4922-b9b5-55e93b3f2453 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:57.322696 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:57.334378 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 300/608] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:35:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:57.335751 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbfc8ac-ecce-4922-b9b5-55e93b3f2453 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:57.410711 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b312e3d9-f5b6-4660-a19f-41a9623042ec None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:57.451278 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85d90e-3941-4f03-9f85-a249be8b6ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:57.452309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85d90e-3941-4f03-9f85-a249be8b6ff5 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:57.453312 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85d90e-3941-4f03-9f85-a249be8b6ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:57.453795 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d85d90e-3941-4f03-9f85-a249be8b6ff5 None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:57.454801 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d85d90e-3941-4f03-9f85-a249be8b6ff5 None admin] RBAC: Adding query filter params (name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:57.455596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d85d90e-3941-4f03-9f85-a249be8b6ff5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:57.457454 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d85d90e-3941-4f03-9f85-a249be8b6ff5 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:57.458169 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbfc8ac-ecce-4922-b9b5-55e93b3f2453 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:57.469139 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 309/609] 10.222.0.115 () {56 vars in 1108 bytes} [Tue Aug 2 10:35:57 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:57.470276 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85d90e-3941-4f03-9f85-a249be8b6ff5 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:57.489060 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:57.497263 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027bf26b-59e6-474b-a554-bf4ce6eaecbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:57.497935 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027bf26b-59e6-474b-a554-bf4ce6eaecbe None admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:57.498208 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027bf26b-59e6-474b-a554-bf4ce6eaecbe None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:57.498476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027bf26b-59e6-474b-a554-bf4ce6eaecbe None admin] PATH_INFO: `/v3/projects/c1eae881e5714bf4bd223809e1a2d920/users/98dcb74152ec4375a586ea526e7f07c8/roles/0f1c61dab22c4d97ae784468058036c7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:57.516735 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027bf26b-59e6-474b-a554-bf4ce6eaecbe None admin] RBAC: Authorizing `identity:create_grant(project_id=c1eae881e5714bf4bd223809e1a2d920, user_id=98dcb74152ec4375a586ea526e7f07c8, role_id=0f1c61dab22c4d97ae784468058036c7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:57.518175 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027bf26b-59e6-474b-a554-bf4ce6eaecbe None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:57.567740 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:57.583829 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d85d90e-3941-4f03-9f85-a249be8b6ff5 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:57.617399 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e3f17b-93fb-4649-ae67-6999a8fcee30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:57.618065 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3f17b-93fb-4649-ae67-6999a8fcee30 None admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:57.618349 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3f17b-93fb-4649-ae67-6999a8fcee30 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:57.621413 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e3f17b-93fb-4649-ae67-6999a8fcee30 None admin] PATH_INFO: `/v3/projects/c1eae881e5714bf4bd223809e1a2d920/users/e358fd7c1b3c43d7ac89fb3562678266/roles/0f1c61dab22c4d97ae784468058036c7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:57.622175 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 301/610] 10.222.0.115 () {58 vars in 1444 bytes} [Tue Aug 2 10:35:57 2022] PUT /identity/v3/projects/c1eae881e5714bf4bd223809e1a2d920/users/98dcb74152ec4375a586ea526e7f07c8/roles/0f1c61dab22c4d97ae784468058036c7 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:35:57.623160 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027bf26b-59e6-474b-a554-bf4ce6eaecbe None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:57.644933 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e3f17b-93fb-4649-ae67-6999a8fcee30 None admin] RBAC: Authorizing `identity:create_grant(project_id=c1eae881e5714bf4bd223809e1a2d920, user_id=e358fd7c1b3c43d7ac89fb3562678266, role_id=0f1c61dab22c4d97ae784468058036c7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:57.647021 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e3f17b-93fb-4649-ae67-6999a8fcee30 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:57.747496 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 310/611] 10.222.0.115 () {58 vars in 1444 bytes} [Tue Aug 2 10:35:57 2022] PUT /identity/v3/projects/c1eae881e5714bf4bd223809e1a2d920/users/e358fd7c1b3c43d7ac89fb3562678266/roles/0f1c61dab22c4d97ae784468058036c7 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:35:57.749709 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157df6b-658e-4cc7-9d80-4e1ac3df9fac None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:57.750076 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157df6b-658e-4cc7-9d80-4e1ac3df9fac None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:57.750419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3157df6b-658e-4cc7-9d80-4e1ac3df9fac None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:57.752669 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 311/612] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:57 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:57.759286 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47101b1b-f732-48a9-83eb-1bb8b91edd77 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:57.759654 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47101b1b-f732-48a9-83eb-1bb8b91edd77 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:57.760007 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47101b1b-f732-48a9-83eb-1bb8b91edd77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:57.763995 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027bf26b-59e6-474b-a554-bf4ce6eaecbe None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:57.795312 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-47101b1b-f732-48a9-83eb-1bb8b91edd77 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:57.798036 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-47101b1b-f732-48a9-83eb-1bb8b91edd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:57.815361 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4a878-6fd5-48f0-8699-ae9724690285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:57.816866 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4a878-6fd5-48f0-8699-ae9724690285 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:57.818334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4a878-6fd5-48f0-8699-ae9724690285 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:57.819086 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc4a878-6fd5-48f0-8699-ae9724690285 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:57.835537 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4a878-6fd5-48f0-8699-ae9724690285 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:57.837824 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc4a878-6fd5-48f0-8699-ae9724690285 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:57.861006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-7dc4a878-6fd5-48f0-8699-ae9724690285 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:57.861006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:57.861006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:57.861006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:57.861006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:57.861006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:57.861006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:57.861006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:57.861006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:57.861006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:57.861006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:57.861006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:57.861006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:35:57.861006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:35:57.861006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:35:57.861970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:35:57.861970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:57.861970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:57.861970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:57.861970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:57.861970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:57.861970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:57.861970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:57.861970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:57.861970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:57.861970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:57.861970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:57.861970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:57.861970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:57.861970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:57.861970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:57.861970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:57.862908 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:35:57.862908 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:35:57.862908 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:57.862908 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:57.862908 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 302/613] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:57.872650 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4a878-6fd5-48f0-8699-ae9724690285 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:57.934498 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 312/614] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:57 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:57.936497 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:57.973956 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47101b1b-f732-48a9-83eb-1bb8b91edd77 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:58.038443 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc4a878-6fd5-48f0-8699-ae9724690285 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:58.055646 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:58.077570 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d5850-cc7c-4d44-8363-35c72644cb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:58.085674 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d5850-cc7c-4d44-8363-35c72644cb2e None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:58.085674 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d5850-cc7c-4d44-8363-35c72644cb2e None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:58.085674 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d5850-cc7c-4d44-8363-35c72644cb2e None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:58.085674 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d5850-cc7c-4d44-8363-35c72644cb2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:58.085674 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d5850-cc7c-4d44-8363-35c72644cb2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:58.085674 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d5850-cc7c-4d44-8363-35c72644cb2e None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:58.090813 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47101b1b-f732-48a9-83eb-1bb8b91edd77 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:58.096193 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 303/615] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:35:57 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:58.101638 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d5850-cc7c-4d44-8363-35c72644cb2e None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:58.131747 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aaf24d-50a6-4347-8b14-dd9c8fe8ef03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:58.132663 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aaf24d-50a6-4347-8b14-dd9c8fe8ef03 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:58.133093 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aaf24d-50a6-4347-8b14-dd9c8fe8ef03 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:58.133600 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aaf24d-50a6-4347-8b14-dd9c8fe8ef03 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:58.145993 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aaf24d-50a6-4347-8b14-dd9c8fe8ef03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:58.151505 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-f4aaf24d-50a6-4347-8b14-dd9c8fe8ef03 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:35:58.151505 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:58.151505 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:58.151505 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:58.151505 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:58.151505 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:58.151505 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:58.151505 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:58.151505 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:58.151505 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:58.151505 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:58.151505 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:58.151505 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:58.151505 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:58.151505 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:58.151505 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:58.159501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:58.159501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:58.159501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:58.159501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:58.159501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:58.159501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:58.159501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:58.159501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:58.159501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:58.159501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:58.159501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:58.159501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:58.159501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:58.159501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:58.159501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:58.159501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:58.160555 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:58.160555 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:58.160555 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:58.160555 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:58.160555 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:58.160555 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:58.160555 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:58.160555 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:58.160555 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:58.160555 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:58.160555 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:35:58.160555 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:58.160555 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 313/616] 10.222.0.115 () {56 vars in 1107 bytes} [Tue Aug 2 10:35:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:58.162603 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aaf24d-50a6-4347-8b14-dd9c8fe8ef03 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:58.200485 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:58.211406 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d5850-cc7c-4d44-8363-35c72644cb2e None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:58.260951 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c9e52-c624-4297-8d32-16c379404889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:58.261653 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c9e52-c624-4297-8d32-16c379404889 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:58.261940 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c9e52-c624-4297-8d32-16c379404889 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:58.262221 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482c9e52-c624-4297-8d32-16c379404889 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:58.280856 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aaf24d-50a6-4347-8b14-dd9c8fe8ef03 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:58.285670 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c9e52-c624-4297-8d32-16c379404889 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:58.287181 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482c9e52-c624-4297-8d32-16c379404889 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:58.311945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-482c9e52-c624-4297-8d32-16c379404889 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:35:58.311945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:58.311945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:58.311945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:58.311945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:58.311945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:58.311945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:58.311945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:58.311945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:58.311945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:58.311945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:58.311945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:58.311945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:35:58.311945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:35:58.311945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:35:58.311945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:35:58.312941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:58.312941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:58.312941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:35:58.312941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:35:58.312941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:35:58.312941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:35:58.312941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:58.312941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:58.312941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:58.312941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:58.312941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:58.312941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:58.312941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:58.312941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:58.312941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:58.312941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:58.312941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:58.312941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:58.313918 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:58.313918 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:58.313918 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:35:58.313918 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:35:58.313918 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:35:58.313918 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:35:58.313918 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:35:58.313918 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:35:58.313918 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:35:58.313918 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:58.314461 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:58.314535 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 304/617] 10.222.0.115 () {56 vars in 1098 bytes} [Tue Aug 2 10:35:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:58.320125 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c9e52-c624-4297-8d32-16c379404889 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:58.323706 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa18f18-c73c-4e03-b0e8-7b3022bb1b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:58.324882 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa18f18-c73c-4e03-b0e8-7b3022bb1b49 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:58.325586 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa18f18-c73c-4e03-b0e8-7b3022bb1b49 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:58.331593 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa18f18-c73c-4e03-b0e8-7b3022bb1b49 None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:58.331593 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa18f18-c73c-4e03-b0e8-7b3022bb1b49 None admin] RBAC: Adding query filter params (name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:58.331593 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa18f18-c73c-4e03-b0e8-7b3022bb1b49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:58.331593 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa18f18-c73c-4e03-b0e8-7b3022bb1b49 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:58.340656 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 314/618] 10.222.0.115 () {56 vars in 1108 bytes} [Tue Aug 2 10:35:58 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:58.347714 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa18f18-c73c-4e03-b0e8-7b3022bb1b49 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:58.458292 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa18f18-c73c-4e03-b0e8-7b3022bb1b49 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:58.496066 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133a022-6ca4-4ccb-9dfe-aa6bfcf11612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:58.497074 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133a022-6ca4-4ccb-9dfe-aa6bfcf11612 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:58.497551 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133a022-6ca4-4ccb-9dfe-aa6bfcf11612 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:58.498059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133a022-6ca4-4ccb-9dfe-aa6bfcf11612 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:58.506335 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482c9e52-c624-4297-8d32-16c379404889 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:58.510477 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133a022-6ca4-4ccb-9dfe-aa6bfcf11612 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:58.512272 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133a022-6ca4-4ccb-9dfe-aa6bfcf11612 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:58.529915 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-d133a022-6ca4-4ccb-9dfe-aa6bfcf11612 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:58.529915 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:58.529915 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:58.529915 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:58.529915 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:58.529915 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:58.529915 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:58.529915 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:58.529915 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:58.529915 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:58.529915 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:58.529915 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:58.529915 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:35:58.529915 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:35:58.529915 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:35:58.531806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:35:58.531806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:58.531806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:58.531806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:58.531806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:58.531806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:58.531806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:58.531806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:58.531806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:58.531806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:58.531806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:58.531806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:58.531806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:58.531806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:58.531806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:58.531806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:58.531806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:58.532713 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:35:58.532713 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:35:58.532713 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:58.532713 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:58.535179 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 315/619] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:58.540750 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133a022-6ca4-4ccb-9dfe-aa6bfcf11612 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:58.560765 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de8f9b-aa24-455a-bc1e-e9f9e2af5933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:58.561420 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de8f9b-aa24-455a-bc1e-e9f9e2af5933 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:58.561701 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de8f9b-aa24-455a-bc1e-e9f9e2af5933 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:58.562029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de8f9b-aa24-455a-bc1e-e9f9e2af5933 None admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:58.562790 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de8f9b-aa24-455a-bc1e-e9f9e2af5933 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:58.563125 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de8f9b-aa24-455a-bc1e-e9f9e2af5933 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:58.564817 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de8f9b-aa24-455a-bc1e-e9f9e2af5933 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:58.584779 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 305/620] 10.222.0.115 () {56 vars in 1138 bytes} [Tue Aug 2 10:35:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:58.590501 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de8f9b-aa24-455a-bc1e-e9f9e2af5933 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:58.594805 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:58.602405 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:58.656154 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133a022-6ca4-4ccb-9dfe-aa6bfcf11612 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:58.696647 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20120610-3d1e-4306-a649-8b9472ef2306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:58.697663 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20120610-3d1e-4306-a649-8b9472ef2306 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:58.698294 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20120610-3d1e-4306-a649-8b9472ef2306 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:58.698763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20120610-3d1e-4306-a649-8b9472ef2306 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:58.699685 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20120610-3d1e-4306-a649-8b9472ef2306 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:58.700210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20120610-3d1e-4306-a649-8b9472ef2306 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:58.702808 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20120610-3d1e-4306-a649-8b9472ef2306 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:58.716221 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 316/621] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:35:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:35:58.722860 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20120610-3d1e-4306-a649-8b9472ef2306 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:58.836079 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:58.847081 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20120610-3d1e-4306-a649-8b9472ef2306 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:58.849068 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de8f9b-aa24-455a-bc1e-e9f9e2af5933 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:58.914902 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250d13b-76c7-44bc-ad91-44c14cbacb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:58.915587 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250d13b-76c7-44bc-ad91-44c14cbacb0c None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:58.915870 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250d13b-76c7-44bc-ad91-44c14cbacb0c None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:58.916144 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7250d13b-76c7-44bc-ad91-44c14cbacb0c None admin] PATH_INFO: `/v3/users/swift` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:58.921125 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022df6e-52b3-4517-9bf0-1a65f96318c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:58.921924 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022df6e-52b3-4517-9bf0-1a65f96318c6 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:58.922214 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022df6e-52b3-4517-9bf0-1a65f96318c6 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:58.922491 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022df6e-52b3-4517-9bf0-1a65f96318c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:58.930440 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250d13b-76c7-44bc-ad91-44c14cbacb0c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:58.936933 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7250d13b-76c7-44bc-ad91-44c14cbacb0c None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:58.957760 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-7250d13b-76c7-44bc-ad91-44c14cbacb0c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:35:58.957760 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:58.957760 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:58.957760 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:58.957760 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:58.957760 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:58.957760 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:58.957760 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:58.957760 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:58.957760 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:58.957760 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:58.957760 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:58.957760 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:35:58.957760 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:35:58.957760 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:35:58.957760 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:35:58.958781 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:58.958781 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:58.958781 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:35:58.958781 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:35:58.958781 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:35:58.958781 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:35:58.958781 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:58.958781 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:58.958781 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:58.958781 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:58.958781 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:58.958781 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:58.958781 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:58.958781 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:58.958781 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:58.958781 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:58.958781 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:58.958781 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:58.959767 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:58.959767 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:58.959767 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:35:58.959767 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:35:58.959767 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:35:58.959767 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:35:58.959767 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:35:58.959767 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:35:58.959767 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:35:58.959767 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:58.965055 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 317/622] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:35:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:35:58.966181 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250d13b-76c7-44bc-ad91-44c14cbacb0c None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:59.072737 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7250d13b-76c7-44bc-ad91-44c14cbacb0c None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:59.076200 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3022df6e-52b3-4517-9bf0-1a65f96318c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:59.077371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3022df6e-52b3-4517-9bf0-1a65f96318c6 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:59.096989 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:59.111818 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c4695-2313-4150-9485-8afaee43adbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:59.115147 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c4695-2313-4150-9485-8afaee43adbb None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:59.115147 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c4695-2313-4150-9485-8afaee43adbb None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:59.115147 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049c4695-2313-4150-9485-8afaee43adbb None admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:59.115147 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049c4695-2313-4150-9485-8afaee43adbb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:59.115147 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049c4695-2313-4150-9485-8afaee43adbb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:59.116232 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049c4695-2313-4150-9485-8afaee43adbb None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:59.136161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 318/623] 10.222.0.115 () {56 vars in 1140 bytes} [Tue Aug 2 10:35:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:59.142152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c4695-2313-4150-9485-8afaee43adbb None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:59.165049 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:59.253016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049c4695-2313-4150-9485-8afaee43adbb None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:59.275521 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 306/624] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:35:58 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 685 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:35:59.276933 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997de44-d523-43e5-99b4-30727e3e04a8 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:59.277217 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997de44-d523-43e5-99b4-30727e3e04a8 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:59.277491 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c997de44-d523-43e5-99b4-30727e3e04a8 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:59.283094 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 307/625] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:35:59 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:35:59.283975 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c997de44-d523-43e5-99b4-30727e3e04a8 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:59.293989 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e9ed5c-c46a-4c0a-801c-2c89e4ad952e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:59.294676 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9ed5c-c46a-4c0a-801c-2c89e4ad952e None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:59.294950 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9ed5c-c46a-4c0a-801c-2c89e4ad952e None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:59.295455 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9ed5c-c46a-4c0a-801c-2c89e4ad952e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:59.310768 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e9ed5c-c46a-4c0a-801c-2c89e4ad952e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:59.313088 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e9ed5c-c46a-4c0a-801c-2c89e4ad952e None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:59.333791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-c4e9ed5c-c46a-4c0a-801c-2c89e4ad952e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:59.333791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:59.333791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:59.333791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:59.333791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:59.333791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:59.333791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:59.333791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:59.333791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:59.333791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:59.333791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:59.333791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:59.333791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:35:59.333791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:35:59.333791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:35:59.334771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:35:59.334771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:59.334771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:59.334771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:59.334771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:59.334771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:59.334771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:59.334771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:59.334771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:59.334771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:59.334771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:59.334771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:59.334771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:59.334771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:59.334771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:59.334771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:59.334771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:59.335707 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:35:59.335707 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:35:59.335707 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:59.335707 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:35:59.335938 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 319/626] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:35:59.337025 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da009c-668e-490e-ba94-899128849567 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:59.339449 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da009c-668e-490e-ba94-899128849567 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:59.339797 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02da009c-668e-490e-ba94-899128849567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:59.361217 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-02da009c-668e-490e-ba94-899128849567 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:35:59.362438 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-02da009c-668e-490e-ba94-899128849567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:35:59.413814 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c997de44-d523-43e5-99b4-30727e3e04a8 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:59.425898 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:59.447417 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:59.455179 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6bff8-57b4-42bd-bc7d-e09a9b0f0cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:59.456086 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6bff8-57b4-42bd-bc7d-e09a9b0f0cd9 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:59.456360 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6bff8-57b4-42bd-bc7d-e09a9b0f0cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:59.456629 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a6bff8-57b4-42bd-bc7d-e09a9b0f0cd9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:59.457900 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 320/627] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:35:59 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:35:59.459088 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02da009c-668e-490e-ba94-899128849567 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:59.467287 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a6bff8-57b4-42bd-bc7d-e09a9b0f0cd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:59.471285 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a6bff8-57b4-42bd-bc7d-e09a9b0f0cd9 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:59.488883 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-43a6bff8-57b4-42bd-bc7d-e09a9b0f0cd9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:59.488883 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:59.488883 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:59.488883 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:59.488883 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:59.488883 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:59.488883 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:59.488883 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:59.488883 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:59.488883 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:59.488883 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:59.488883 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:59.488883 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:35:59.488883 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:35:59.488883 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:35:59.490029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:35:59.490029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:59.490029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:59.490029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:59.490029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:59.490029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:59.490029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:59.490029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:59.490029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:59.490029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:59.490029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:59.490029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:59.490029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:59.490029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:59.490029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:59.490029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:59.490029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:59.491246 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:35:59.491246 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:35:59.491246 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:35:59.491246 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:59.491246 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 308/628] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:35:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:59.495237 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6bff8-57b4-42bd-bc7d-e09a9b0f0cd9 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:59.584447 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02da009c-668e-490e-ba94-899128849567 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:59.621998 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d255824-92cd-444e-8943-32b820292482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:59.623444 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d255824-92cd-444e-8943-32b820292482 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:59.623994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d255824-92cd-444e-8943-32b820292482 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:59.624808 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d255824-92cd-444e-8943-32b820292482 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:59.625728 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d255824-92cd-444e-8943-32b820292482 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:59.626246 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d255824-92cd-444e-8943-32b820292482 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:59.627842 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d255824-92cd-444e-8943-32b820292482 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:59.634605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a6bff8-57b4-42bd-bc7d-e09a9b0f0cd9 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:59.647408 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 321/629] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:35:59 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:59.648658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d255824-92cd-444e-8943-32b820292482 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:59.677396 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b1eee-73a8-4e47-892e-b63d4bd97561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:59.678387 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b1eee-73a8-4e47-892e-b63d4bd97561 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:59.678896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b1eee-73a8-4e47-892e-b63d4bd97561 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:59.679452 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354b1eee-73a8-4e47-892e-b63d4bd97561 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:59.692443 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354b1eee-73a8-4e47-892e-b63d4bd97561 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:35:59.698547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-354b1eee-73a8-4e47-892e-b63d4bd97561 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:35:59.698547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:59.698547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:59.698547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:59.698547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:59.698547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:59.698547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:59.698547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:59.698547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:59.698547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:59.698547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:59.698547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:59.698547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:35:59.698547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:35:59.698547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:35:59.698547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:35:59.699760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:35:59.699760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:35:59.699760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:35:59.699760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:35:59.699760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:59.699760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:59.699760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:59.699760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:59.699760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:59.699760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:59.699760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:59.699760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:59.699760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:59.699760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:59.699760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:59.699760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:59.700599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:59.700599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:59.700599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:59.700599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:59.700599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:35:59.700599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:35:59.700599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:35:59.700599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:35:59.700599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:35:59.700599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:35:59.700599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:35:59.700599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:59.703907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 309/630] 10.222.0.115 () {56 vars in 1107 bytes} [Tue Aug 2 10:35:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:59.705082 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b1eee-73a8-4e47-892e-b63d4bd97561 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:59.736418 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:59.738675 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:35:59.785694 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d255824-92cd-444e-8943-32b820292482 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:59.824530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bbc6d-a927-4ff9-8863-2ef2ac8b2a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:59.825733 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bbc6d-a927-4ff9-8863-2ef2ac8b2a22 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:59.826279 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bbc6d-a927-4ff9-8863-2ef2ac8b2a22 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:59.827301 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853bbc6d-a927-4ff9-8863-2ef2ac8b2a22 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:59.828304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853bbc6d-a927-4ff9-8863-2ef2ac8b2a22 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:35:59.828913 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853bbc6d-a927-4ff9-8863-2ef2ac8b2a22 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:59.830984 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853bbc6d-a927-4ff9-8863-2ef2ac8b2a22 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:59.836243 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354b1eee-73a8-4e47-892e-b63d4bd97561 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:35:59.846208 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 322/631] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:35:59 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:35:59.847673 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bbc6d-a927-4ff9-8863-2ef2ac8b2a22 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:59.879114 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7b537-3a12-4d5c-beb7-6011e79e14dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:35:59.879776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7b537-3a12-4d5c-beb7-6011e79e14dd None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:35:59.880055 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7b537-3a12-4d5c-beb7-6011e79e14dd None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:35:59.880326 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7b537-3a12-4d5c-beb7-6011e79e14dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:35:59.890253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e7b537-3a12-4d5c-beb7-6011e79e14dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:35:59.891602 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e7b537-3a12-4d5c-beb7-6011e79e14dd None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:35:59.905689 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-68e7b537-3a12-4d5c-beb7-6011e79e14dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:35:59.905689 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:35:59.905689 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:35:59.905689 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:35:59.905689 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:35:59.905689 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:35:59.905689 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:35:59.905689 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:35:59.905689 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:35:59.905689 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:35:59.905689 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:35:59.905689 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:35:59.905689 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:35:59.905689 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:35:59.905689 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:35:59.906750 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:35:59.906750 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:35:59.906750 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:35:59.906750 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:35:59.906750 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:35:59.906750 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:35:59.906750 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:35:59.906750 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:35:59.906750 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:35:59.906750 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:35:59.906750 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:35:59.906750 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:35:59.906750 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:35:59.906750 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:35:59.906750 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:35:59.906750 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:35:59.906750 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:35:59.908248 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:35:59.908248 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:35:59.908248 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:35:59.908248 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:35:59.908248 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:35:59.908248 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:35:59.908248 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:35:59.908248 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:35:59.911114 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 310/632] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:35:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:35:59.912292 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7b537-3a12-4d5c-beb7-6011e79e14dd None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:35:59.978565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853bbc6d-a927-4ff9-8863-2ef2ac8b2a22 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:00.006194 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:00.022176 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6c933-45be-497b-be1f-c6d4c1763a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:00.022797 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6c933-45be-497b-be1f-c6d4c1763a84 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:00.023092 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6c933-45be-497b-be1f-c6d4c1763a84 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:00.023368 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb6c933-45be-497b-be1f-c6d4c1763a84 None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:00.024082 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6c933-45be-497b-be1f-c6d4c1763a84 None admin] RBAC: Adding query filter params (name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:00.024439 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6c933-45be-497b-be1f-c6d4c1763a84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:00.024698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:00.026413 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb6c933-45be-497b-be1f-c6d4c1763a84 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:00.039264 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 323/633] 10.222.0.115 () {56 vars in 1108 bytes} [Tue Aug 2 10:35:59 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:00.039264 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6c933-45be-497b-be1f-c6d4c1763a84 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:00.054702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7b537-3a12-4d5c-beb7-6011e79e14dd None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:00.102137 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99663df-779f-4c1c-b3f0-9ea01f325bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:00.103882 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99663df-779f-4c1c-b3f0-9ea01f325bf4 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:00.104628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99663df-779f-4c1c-b3f0-9ea01f325bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:00.105511 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99663df-779f-4c1c-b3f0-9ea01f325bf4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:00.116436 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99663df-779f-4c1c-b3f0-9ea01f325bf4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:00.119106 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99663df-779f-4c1c-b3f0-9ea01f325bf4 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:00.135067 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-c99663df-779f-4c1c-b3f0-9ea01f325bf4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:00.135067 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:00.135067 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:00.135067 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:00.135067 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:00.135067 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:00.135067 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:00.135067 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:00.135067 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:00.135067 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:00.135067 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:00.135067 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:00.135067 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:36:00.135067 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:36:00.135067 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:36:00.135993 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:36:00.135993 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:00.135993 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:00.135993 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:00.135993 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:00.135993 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:00.135993 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:00.135993 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:00.135993 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:00.135993 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:00.135993 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:00.135993 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:00.135993 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:00.135993 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:00.135993 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:00.135993 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:00.135993 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:00.136876 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:36:00.136876 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:36:00.136876 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:36:00.136876 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:36:00.136876 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:36:00.136876 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:36:00.136876 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:00.136876 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:00.144349 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 311/634] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:35:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:00.144349 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99663df-779f-4c1c-b3f0-9ea01f325bf4 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:00.164244 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb6c933-45be-497b-be1f-c6d4c1763a84 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:00.209239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d69fcd-7773-4dda-b0ef-c76625ea0a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:00.210172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d69fcd-7773-4dda-b0ef-c76625ea0a50 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:00.210648 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d69fcd-7773-4dda-b0ef-c76625ea0a50 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:00.211107 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d69fcd-7773-4dda-b0ef-c76625ea0a50 None admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:00.212362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d69fcd-7773-4dda-b0ef-c76625ea0a50 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:00.212905 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d69fcd-7773-4dda-b0ef-c76625ea0a50 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:00.214664 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d69fcd-7773-4dda-b0ef-c76625ea0a50 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:00.235171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 324/635] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:36:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:00.235171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d69fcd-7773-4dda-b0ef-c76625ea0a50 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:00.270600 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99663df-779f-4c1c-b3f0-9ea01f325bf4 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:00.281577 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:00.306756 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec421014-a21c-427d-a854-03c2368edb1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:00.307421 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec421014-a21c-427d-a854-03c2368edb1c None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:00.307700 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec421014-a21c-427d-a854-03c2368edb1c None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:00.307969 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec421014-a21c-427d-a854-03c2368edb1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:00.317675 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:00.319954 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec421014-a21c-427d-a854-03c2368edb1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:00.321428 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec421014-a21c-427d-a854-03c2368edb1c None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:00.337050 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-ec421014-a21c-427d-a854-03c2368edb1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:00.337050 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:00.337050 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:00.337050 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:00.337050 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:00.337050 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:00.337050 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:00.337050 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:00.337050 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:00.337050 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:00.337050 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:00.337050 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:00.337050 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:00.337050 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:00.337050 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:00.349113 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:00.349113 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:00.349113 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:00.349113 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:00.349113 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:00.349113 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:00.349113 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:00.349113 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:00.349113 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:00.349113 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:00.349113 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:00.349113 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:00.349113 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:00.349113 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:00.349113 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:00.349113 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:00.349113 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:00.350273 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:00.350273 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:00.350273 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:00.350273 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:00.350273 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 312/636] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:00.350273 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c0ff2-4236-431c-8ed5-4358ab070dea None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:00.350273 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c0ff2-4236-431c-8ed5-4358ab070dea None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:00.350273 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6c0ff2-4236-431c-8ed5-4358ab070dea None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:00.350273 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 313/637] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:00 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:00.350828 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c0ff2-4236-431c-8ed5-4358ab070dea None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:00.350900 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d69fcd-7773-4dda-b0ef-c76625ea0a50 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:00.381037 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbdf128-5169-4291-869a-e248b3e4c144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:00.381663 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbdf128-5169-4291-869a-e248b3e4c144 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:00.381968 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbdf128-5169-4291-869a-e248b3e4c144 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:00.382245 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbdf128-5169-4291-869a-e248b3e4c144 None admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:00.383008 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbdf128-5169-4291-869a-e248b3e4c144 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:00.383336 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbdf128-5169-4291-869a-e248b3e4c144 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:00.384868 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbdf128-5169-4291-869a-e248b3e4c144 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:00.397450 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 325/638] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:36:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:00.398672 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef8b1e-f993-4d1e-8f3e-e3d913268c62 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:00.398965 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef8b1e-f993-4d1e-8f3e-e3d913268c62 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:00.399275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef8b1e-f993-4d1e-8f3e-e3d913268c62 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:00.400505 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 326/639] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:00 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:00.401195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef8b1e-f993-4d1e-8f3e-e3d913268c62 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:00.454223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6c0ff2-4236-431c-8ed5-4358ab070dea None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:00.488870 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbba039-9371-46ab-a70c-3dc15290bbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:00.489592 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbba039-9371-46ab-a70c-3dc15290bbfa None admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:00.490800 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbba039-9371-46ab-a70c-3dc15290bbfa None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:00.490800 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbba039-9371-46ab-a70c-3dc15290bbfa None admin] PATH_INFO: `/v3/projects/c1eae881e5714bf4bd223809e1a2d920/users/0daa49b5a35f4a468bb8d0939e7ec20c/roles/0f1c61dab22c4d97ae784468058036c7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:00.506254 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbba039-9371-46ab-a70c-3dc15290bbfa None admin] RBAC: Authorizing `identity:create_grant(project_id=c1eae881e5714bf4bd223809e1a2d920, user_id=0daa49b5a35f4a468bb8d0939e7ec20c, role_id=0f1c61dab22c4d97ae784468058036c7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:00.507780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbba039-9371-46ab-a70c-3dc15290bbfa None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:00.513091 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef8b1e-f993-4d1e-8f3e-e3d913268c62 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:00.513091 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:00.543643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4be80a-d7e8-4e86-b661-37caaa56a0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:00.544249 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4be80a-d7e8-4e86-b661-37caaa56a0b1 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:00.544530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4be80a-d7e8-4e86-b661-37caaa56a0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:00.544801 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4be80a-d7e8-4e86-b661-37caaa56a0b1 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:00.545584 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4be80a-d7e8-4e86-b661-37caaa56a0b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:00.545887 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4be80a-d7e8-4e86-b661-37caaa56a0b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:00.547267 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4be80a-d7e8-4e86-b661-37caaa56a0b1 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:00.558342 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 327/640] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:00 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:00.559578 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f092ab-4fbd-49e7-a8a7-4bda0f90e797 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:00.559867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f092ab-4fbd-49e7-a8a7-4bda0f90e797 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:00.561283 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f092ab-4fbd-49e7-a8a7-4bda0f90e797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:00.578073 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-36f092ab-4fbd-49e7-a8a7-4bda0f90e797 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:00.579292 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-36f092ab-4fbd-49e7-a8a7-4bda0f90e797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:00.609484 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 314/641] 10.222.0.115 () {58 vars in 1444 bytes} [Tue Aug 2 10:36:00 2022] PUT /identity/v3/projects/c1eae881e5714bf4bd223809e1a2d920/users/0daa49b5a35f4a468bb8d0939e7ec20c/roles/0f1c61dab22c4d97ae784468058036c7 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:36:00.610474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbba039-9371-46ab-a70c-3dc15290bbfa None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:00.657000 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 328/642] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:00 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:36:00.660592 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd5ae0-fda5-4e8c-bef8-9db2b4155ff6 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:00.661436 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd5ae0-fda5-4e8c-bef8-9db2b4155ff6 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:00.661864 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedd5ae0-fda5-4e8c-bef8-9db2b4155ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:00.676717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:00.687588 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-dedd5ae0-fda5-4e8c-bef8-9db2b4155ff6 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:00.690956 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-dedd5ae0-fda5-4e8c-bef8-9db2b4155ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:00.749597 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbba039-9371-46ab-a70c-3dc15290bbfa None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:00.776324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 329/643] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:00 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:36:00.777195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd5ae0-fda5-4e8c-bef8-9db2b4155ff6 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:00.795975 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46da43-4321-4f1e-bd06-a6cf8875895c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:00.796624 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46da43-4321-4f1e-bd06-a6cf8875895c None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:00.796908 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46da43-4321-4f1e-bd06-a6cf8875895c None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:00.797180 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb46da43-4321-4f1e-bd06-a6cf8875895c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:00.798094 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb46da43-4321-4f1e-bd06-a6cf8875895c None admin] RBAC: Adding query filter params (role.id=0f1c61dab22c4d97ae784468058036c7, scope.project.id=c1eae881e5714bf4bd223809e1a2d920, user.id=d0478bd540f446ec810e2091c46ac49c) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:00.799086 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb46da43-4321-4f1e-bd06-a6cf8875895c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:00.799876 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb46da43-4321-4f1e-bd06-a6cf8875895c None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:00.807104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 315/644] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:36:00 2022] GET /identity/v3/role_assignments?user.id=d0478bd540f446ec810e2091c46ac49c&scope.project.id=c1eae881e5714bf4bd223809e1a2d920&role.id=0f1c61dab22c4d97ae784468058036c7 => generated 610 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:00.807404 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46da43-4321-4f1e-bd06-a6cf8875895c None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:00.815362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:00.931268 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedd5ae0-fda5-4e8c-bef8-9db2b4155ff6 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:00.970130 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb46da43-4321-4f1e-bd06-a6cf8875895c None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:00.978272 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:00.982806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34d3cf-eaf4-4007-880c-70c2d1ecfbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:00.982806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34d3cf-eaf4-4007-880c-70c2d1ecfbfb None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:00.982806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34d3cf-eaf4-4007-880c-70c2d1ecfbfb None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:00.982806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a34d3cf-eaf4-4007-880c-70c2d1ecfbfb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:00.991363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a34d3cf-eaf4-4007-880c-70c2d1ecfbfb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:00.992823 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a34d3cf-eaf4-4007-880c-70c2d1ecfbfb None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:01.009685 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-5a34d3cf-eaf4-4007-880c-70c2d1ecfbfb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 10:36:01.009685 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:01.009685 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:01.009685 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:01.009685 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:01.009685 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:01.009685 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:01.009685 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:01.009685 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:01.009685 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:01.009685 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:01.009685 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:01.009685 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:36:01.009685 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:36:01.009685 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:36:01.009685 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:36:01.010798 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:01.010798 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:01.010798 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:36:01.010798 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:01.010798 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:36:01.010798 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:01.010798 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:01.010798 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:01.010798 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:01.010798 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:01.010798 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:01.010798 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:01.010798 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:01.010798 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:01.010798 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:01.010798 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:01.010798 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:01.010798 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:01.011847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:01.011847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:01.011847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:36:01.011847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:36:01.011847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:36:01.011847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:36:01.011847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:36:01.011847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:36:01.011847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 10:36:01.011847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:01.011847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 330/645] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:36:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:01.011847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34d3cf-eaf4-4007-880c-70c2d1ecfbfb None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:01.023605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127b6c1-dbe1-4215-b276-67a8a6c682bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:01.024398 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127b6c1-dbe1-4215-b276-67a8a6c682bb None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:01.024509 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127b6c1-dbe1-4215-b276-67a8a6c682bb None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:01.024776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f127b6c1-dbe1-4215-b276-67a8a6c682bb None admin] PATH_INFO: `/v3/roles/service` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:01.048134 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f127b6c1-dbe1-4215-b276-67a8a6c682bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:36:01.055922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-f127b6c1-dbe1-4215-b276-67a8a6c682bb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:36:01.055922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:01.055922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:01.055922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:01.055922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:01.055922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:01.055922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:01.055922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:01.055922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:01.055922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:01.055922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:01.055922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:01.055922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:36:01.055922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:36:01.055922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:36:01.055922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:36:01.057743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:36:01.057743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:36:01.057743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:36:01.057743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:36:01.057743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:01.057743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:01.057743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:01.057743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:01.057743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:01.057743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:01.057743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:01.057743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:01.057743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:01.057743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:01.057743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:01.057743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:01.059583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:01.059583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:01.059583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:01.059583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:01.059583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:36:01.059583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:36:01.059583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:36:01.059583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:36:01.059583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:36:01.059583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:36:01.059583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:36:01.059583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:01.059583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 316/646] 10.222.0.115 () {56 vars in 1107 bytes} [Tue Aug 2 10:36:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:01.060969 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127b6c1-dbe1-4215-b276-67a8a6c682bb None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:01.150338 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:01.165874 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a34d3cf-eaf4-4007-880c-70c2d1ecfbfb None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:01.204036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f127b6c1-dbe1-4215-b276-67a8a6c682bb None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:01.211347 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fcf67d-d52d-4a24-bd1d-39e3b8f79d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:01.212611 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fcf67d-d52d-4a24-bd1d-39e3b8f79d04 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:01.213073 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fcf67d-d52d-4a24-bd1d-39e3b8f79d04 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:01.213489 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fcf67d-d52d-4a24-bd1d-39e3b8f79d04 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:01.227577 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fcf67d-d52d-4a24-bd1d-39e3b8f79d04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:36:01.235466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-32fcf67d-d52d-4a24-bd1d-39e3b8f79d04 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:36:01.235466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:01.235466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:01.235466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:01.235466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:01.235466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:01.235466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:01.235466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:01.235466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:01.235466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:01.235466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:01.235466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:01.235466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:36:01.235466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:36:01.235466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:36:01.235466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:36:01.236412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:36:01.236412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:36:01.236412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:36:01.236412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:36:01.236412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:01.236412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:01.236412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:01.236412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:01.236412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:01.236412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:01.236412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:01.236412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:01.236412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:01.236412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:01.236412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:01.236412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:01.237275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:01.237275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:01.237275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:01.237275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:01.237275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:36:01.237275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:36:01.237275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:36:01.237275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:36:01.237275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:36:01.237275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:36:01.237275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:36:01.237275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:01.244384 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 331/647] 10.222.0.115 () {56 vars in 1107 bytes} [Tue Aug 2 10:36:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:01.244384 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fcf67d-d52d-4a24-bd1d-39e3b8f79d04 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:01.251765 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd93d715-13e7-4b17-88dc-b1367006c171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:01.252923 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93d715-13e7-4b17-88dc-b1367006c171 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:01.253498 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93d715-13e7-4b17-88dc-b1367006c171 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:01.254074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd93d715-13e7-4b17-88dc-b1367006c171 None admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:01.255522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd93d715-13e7-4b17-88dc-b1367006c171 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:01.255935 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd93d715-13e7-4b17-88dc-b1367006c171 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:01.258027 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd93d715-13e7-4b17-88dc-b1367006c171 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:01.283148 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 317/648] 10.222.0.115 () {56 vars in 1142 bytes} [Tue Aug 2 10:36:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 374 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:01.284246 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd93d715-13e7-4b17-88dc-b1367006c171 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:01.312076 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:01.375026 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fcf67d-d52d-4a24-bd1d-39e3b8f79d04 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:01.394805 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd93d715-13e7-4b17-88dc-b1367006c171 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:01.412946 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e3cba-39a4-498e-8ead-45a1a28c8c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:01.413815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e3cba-39a4-498e-8ead-45a1a28c8c20 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:01.414076 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e3cba-39a4-498e-8ead-45a1a28c8c20 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:01.414518 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988e3cba-39a4-498e-8ead-45a1a28c8c20 None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:01.415389 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e3cba-39a4-498e-8ead-45a1a28c8c20 None admin] RBAC: Adding query filter params (name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:01.415979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e3cba-39a4-498e-8ead-45a1a28c8c20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:01.417444 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988e3cba-39a4-498e-8ead-45a1a28c8c20 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:01.427359 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 332/649] 10.222.0.115 () {56 vars in 1108 bytes} [Tue Aug 2 10:36:01 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:01.428517 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e3cba-39a4-498e-8ead-45a1a28c8c20 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:01.432513 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:01.448557 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6bd4bc-9f49-4589-bbf6-d5e0919ecde4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:01.449418 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6bd4bc-9f49-4589-bbf6-d5e0919ecde4 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:01.449841 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6bd4bc-9f49-4589-bbf6-d5e0919ecde4 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:01.450255 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6bd4bc-9f49-4589-bbf6-d5e0919ecde4 None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:01.451157 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6bd4bc-9f49-4589-bbf6-d5e0919ecde4 None admin] RBAC: Adding query filter params (name=service) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:01.451600 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6bd4bc-9f49-4589-bbf6-d5e0919ecde4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:01.453318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6bd4bc-9f49-4589-bbf6-d5e0919ecde4 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:01.462547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 318/650] 10.222.0.115 () {56 vars in 1108 bytes} [Tue Aug 2 10:36:01 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:01.469351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6bd4bc-9f49-4589-bbf6-d5e0919ecde4 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:01.578087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:01.588494 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988e3cba-39a4-498e-8ead-45a1a28c8c20 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:01.596968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6bd4bc-9f49-4589-bbf6-d5e0919ecde4 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:01.634217 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cfebfa-891e-4e79-9c9d-2e50e2ec1483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:01.634859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfebfa-891e-4e79-9c9d-2e50e2ec1483 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:01.635150 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfebfa-891e-4e79-9c9d-2e50e2ec1483 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:01.635431 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfebfa-891e-4e79-9c9d-2e50e2ec1483 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:01.640123 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3177fff-f6be-4b30-9e31-5ccb37d8fbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:01.641011 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3177fff-f6be-4b30-9e31-5ccb37d8fbb3 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:01.641516 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3177fff-f6be-4b30-9e31-5ccb37d8fbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:01.642011 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3177fff-f6be-4b30-9e31-5ccb37d8fbb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:01.650090 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cfebfa-891e-4e79-9c9d-2e50e2ec1483 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:01.651585 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cfebfa-891e-4e79-9c9d-2e50e2ec1483 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:01.673720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-69cfebfa-891e-4e79-9c9d-2e50e2ec1483 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:01.673720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:01.673720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:01.673720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:01.673720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:01.673720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:01.673720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:01.673720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:01.673720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:01.673720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:01.673720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:01.673720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:01.673720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:01.673720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:01.673720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:01.674690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:01.674690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:01.674690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:01.674690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:01.674690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:01.674690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:01.674690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:01.674690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:01.674690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:01.674690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:01.674690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:01.674690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:01.674690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:01.674690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:01.674690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:01.674690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:01.674690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:01.676177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:01.676177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:01.676177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:01.676177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:01.676177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 319/651] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:01.676177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cfebfa-891e-4e79-9c9d-2e50e2ec1483 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:01.753278 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3177fff-f6be-4b30-9e31-5ccb37d8fbb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:01.756515 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3177fff-f6be-4b30-9e31-5ccb37d8fbb3 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:01.761112 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:01.821136 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cfebfa-891e-4e79-9c9d-2e50e2ec1483 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:01.865863 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071bb6e3-da25-48d2-bb35-3ea6d350fc0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:01.866557 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bb6e3-da25-48d2-bb35-3ea6d350fc0c None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:01.866838 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bb6e3-da25-48d2-bb35-3ea6d350fc0c None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:01.867135 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071bb6e3-da25-48d2-bb35-3ea6d350fc0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:01.875954 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:01.880279 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071bb6e3-da25-48d2-bb35-3ea6d350fc0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:01.881809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071bb6e3-da25-48d2-bb35-3ea6d350fc0c None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:01.897041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-071bb6e3-da25-48d2-bb35-3ea6d350fc0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:01.897041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:01.897041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:01.897041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:01.897041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:01.897041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:01.897041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:01.897041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:01.897041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:01.897041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:01.897041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:01.897041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:01.897041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:01.897041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:01.897041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:01.898010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:01.898010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:01.898010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:01.898010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:01.898010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:01.898010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:01.898010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:01.898010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:01.898010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:01.898010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:01.898010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:01.898010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:01.898010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:01.898010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:01.898010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:01.898010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:01.898010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:01.899003 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:01.899003 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:01.899003 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:01.899003 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:01.903064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 320/652] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:01.903928 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071bb6e3-da25-48d2-bb35-3ea6d350fc0c None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:01.911317 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 333/653] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:36:01 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:36:01.913817 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3177fff-f6be-4b30-9e31-5ccb37d8fbb3 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:02.016679 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071bb6e3-da25-48d2-bb35-3ea6d350fc0c None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:02.031946 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3177fff-f6be-4b30-9e31-5ccb37d8fbb3 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:02.034824 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:02.055403 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9105e11-605c-4a8f-8cb3-c51c21e203a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:02.056019 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9105e11-605c-4a8f-8cb3-c51c21e203a1 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:02.056291 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9105e11-605c-4a8f-8cb3-c51c21e203a1 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:02.056561 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9105e11-605c-4a8f-8cb3-c51c21e203a1 None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:02.057424 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9105e11-605c-4a8f-8cb3-c51c21e203a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:02.057534 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9105e11-605c-4a8f-8cb3-c51c21e203a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:02.058901 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9105e11-605c-4a8f-8cb3-c51c21e203a1 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:02.073207 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 321/654] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:02.074168 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9105e11-605c-4a8f-8cb3-c51c21e203a1 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:02.074441 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fadbd66-d7b1-4060-9dab-e7601ae41560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:02.075391 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadbd66-d7b1-4060-9dab-e7601ae41560 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:02.075896 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadbd66-d7b1-4060-9dab-e7601ae41560 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:02.076386 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fadbd66-d7b1-4060-9dab-e7601ae41560 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:02.077946 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fadbd66-d7b1-4060-9dab-e7601ae41560 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:02.078531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fadbd66-d7b1-4060-9dab-e7601ae41560 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:02.080389 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fadbd66-d7b1-4060-9dab-e7601ae41560 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:02.094409 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 334/655] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:02.095647 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fadbd66-d7b1-4060-9dab-e7601ae41560 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:02.132398 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:02.192713 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9105e11-605c-4a8f-8cb3-c51c21e203a1 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:02.220215 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fadbd66-d7b1-4060-9dab-e7601ae41560 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:02.240614 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7842e-6d4a-4405-9290-26378a1d17d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:02.242583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7842e-6d4a-4405-9290-26378a1d17d0 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:02.243086 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7842e-6d4a-4405-9290-26378a1d17d0 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:02.243570 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a7842e-6d4a-4405-9290-26378a1d17d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:02.259375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7842e-6d4a-4405-9290-26378a1d17d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:02.260831 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a7842e-6d4a-4405-9290-26378a1d17d0 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:02.266203 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c440778-dd7b-4c7f-9e04-5a917bf239de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:02.267251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c440778-dd7b-4c7f-9e04-5a917bf239de None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:02.267750 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c440778-dd7b-4c7f-9e04-5a917bf239de None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:02.268220 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c440778-dd7b-4c7f-9e04-5a917bf239de None admin] PATH_INFO: `/v3/users/neutron` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:02.277149 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-68a7842e-6d4a-4405-9290-26378a1d17d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:02.277149 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:02.277149 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:02.277149 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:02.277149 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:02.277149 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:02.277149 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:02.277149 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:02.277149 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:02.277149 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:02.277149 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:02.277149 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:02.277149 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:02.277149 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:02.277149 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:02.278071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:02.278071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:02.278071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:02.278071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:02.278071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:02.278071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:02.278071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:02.278071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:02.278071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:02.278071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:02.278071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:02.278071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:02.278071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:02.278071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:02.278071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:02.278071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:02.278071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:02.278968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:02.278968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:02.278968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:02.278968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:02.280839 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c440778-dd7b-4c7f-9e04-5a917bf239de None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:02.282683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c440778-dd7b-4c7f-9e04-5a917bf239de None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:02.283330 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 322/656] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:02.284209 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7842e-6d4a-4405-9290-26378a1d17d0 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:02.297225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-5c440778-dd7b-4c7f-9e04-5a917bf239de None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 10:36:02.297225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:02.297225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:02.297225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:02.297225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:02.297225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:02.297225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:02.297225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:02.297225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:02.297225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:02.297225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:02.297225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:02.297225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:36:02.297225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:36:02.297225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:36:02.297225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:36:02.298199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:02.298199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:02.298199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:36:02.298199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:02.298199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:36:02.298199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:02.298199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:02.298199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:02.298199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:02.298199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:02.298199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:02.298199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:02.298199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:02.298199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:02.298199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:02.298199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:02.298199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:02.298199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:02.299326 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:02.299326 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:02.299326 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:36:02.299326 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:36:02.299326 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:36:02.299326 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:36:02.299326 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:36:02.299326 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:36:02.299326 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 02 10:36:02.299326 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:02.302000 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 335/657] 10.222.0.115 () {56 vars in 1107 bytes} [Tue Aug 2 10:36:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:02.303231 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c440778-dd7b-4c7f-9e04-5a917bf239de None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:02.331713 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:02.396593 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a7842e-6d4a-4405-9290-26378a1d17d0 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:02.398124 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:02.413187 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c440778-dd7b-4c7f-9e04-5a917bf239de None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:02.434796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a75a47-24cf-4b27-b6f5-46b3b6802d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:02.435795 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a75a47-24cf-4b27-b6f5-46b3b6802d2a None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:02.435938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a75a47-24cf-4b27-b6f5-46b3b6802d2a None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:02.436741 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a75a47-24cf-4b27-b6f5-46b3b6802d2a None admin] PATH_INFO: `/v3/users/glance` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:02.447082 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a75a47-24cf-4b27-b6f5-46b3b6802d2a None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:02.448499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a75a47-24cf-4b27-b6f5-46b3b6802d2a None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:02.459267 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c9409-320f-4de7-bdff-95da3f77f704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:02.460198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c9409-320f-4de7-bdff-95da3f77f704 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:02.460699 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c9409-320f-4de7-bdff-95da3f77f704 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:02.461175 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33c9409-320f-4de7-bdff-95da3f77f704 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:02.462066 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33c9409-320f-4de7-bdff-95da3f77f704 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:02.462282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-55a75a47-24cf-4b27-b6f5-46b3b6802d2a None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 02 10:36:02.462282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:02.462282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:02.462282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:02.462282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:02.462282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:02.462282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:02.462282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:02.462282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:02.462282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:02.462282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:02.462282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:02.462282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:36:02.462282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:36:02.462282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:36:02.462282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:36:02.463196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:02.463196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:02.463196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:36:02.463196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:02.463196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:36:02.463196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:02.463196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:02.463196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:02.463196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:02.463196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:02.463196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:02.463196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:02.463196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:02.463196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:02.463196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:02.463196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:02.463196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:02.463196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:02.464159 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:02.464159 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:02.464159 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:36:02.464159 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:36:02.464159 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:36:02.464159 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:36:02.464159 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:36:02.464159 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:36:02.464159 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 02 10:36:02.464159 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:02.465457 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33c9409-320f-4de7-bdff-95da3f77f704 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:02.465457 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33c9409-320f-4de7-bdff-95da3f77f704 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:02.468440 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 323/658] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:36:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:02.469566 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a75a47-24cf-4b27-b6f5-46b3b6802d2a None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:02.482585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 336/659] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:02 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:02.484653 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c9409-320f-4de7-bdff-95da3f77f704 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:02.583225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:02.587586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a75a47-24cf-4b27-b6f5-46b3b6802d2a None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:02.602457 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33c9409-320f-4de7-bdff-95da3f77f704 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:02.625753 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc000f86-8715-4019-87b1-40f75e6283ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:02.626376 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc000f86-8715-4019-87b1-40f75e6283ef None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:02.626677 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc000f86-8715-4019-87b1-40f75e6283ef None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:02.626969 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc000f86-8715-4019-87b1-40f75e6283ef None admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:02.627751 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc000f86-8715-4019-87b1-40f75e6283ef None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:02.628048 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc000f86-8715-4019-87b1-40f75e6283ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:02.629475 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc000f86-8715-4019-87b1-40f75e6283ef None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:02.646516 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc6332-fbc7-492b-9aa7-6d7f6bc650a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:02.647555 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc6332-fbc7-492b-9aa7-6d7f6bc650a2 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:02.648035 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc6332-fbc7-492b-9aa7-6d7f6bc650a2 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:02.648477 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 324/660] 10.222.0.115 () {56 vars in 1144 bytes} [Tue Aug 2 10:36:02 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 376 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:02.648855 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dc6332-fbc7-492b-9aa7-6d7f6bc650a2 None admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:02.649382 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc000f86-8715-4019-87b1-40f75e6283ef None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:02.650338 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc6332-fbc7-492b-9aa7-6d7f6bc650a2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:02.650892 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc6332-fbc7-492b-9aa7-6d7f6bc650a2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:02.652954 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dc6332-fbc7-492b-9aa7-6d7f6bc650a2 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:02.673478 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:02.679130 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 337/661] 10.222.0.115 () {56 vars in 1142 bytes} [Tue Aug 2 10:36:02 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:02.680311 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc6332-fbc7-492b-9aa7-6d7f6bc650a2 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:02.761226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc000f86-8715-4019-87b1-40f75e6283ef None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:02.802619 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb11cd-e7f8-4e1d-87cf-3b68e722e8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:02.803729 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb11cd-e7f8-4e1d-87cf-3b68e722e8f7 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:02.804057 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb11cd-e7f8-4e1d-87cf-3b68e722e8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:02.804341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adb11cd-e7f8-4e1d-87cf-3b68e722e8f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:02.812610 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb11cd-e7f8-4e1d-87cf-3b68e722e8f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:02.814148 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adb11cd-e7f8-4e1d-87cf-3b68e722e8f7 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:02.827029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-4adb11cd-e7f8-4e1d-87cf-3b68e722e8f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:02.827029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:02.827029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:02.827029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:02.827029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:02.827029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:02.827029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:02.827029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:02.827029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:02.827029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:02.827029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:02.827029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:02.827029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:36:02.827029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:36:02.827029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:36:02.828087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:36:02.828087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:02.828087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:02.828087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:02.828087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:02.828087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:02.828087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:02.828087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:02.828087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:02.828087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:02.828087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:02.828087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:02.828087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:02.828087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:02.828087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:02.828087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:02.828087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:02.828995 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:36:02.828995 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:36:02.828995 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:36:02.828995 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:36:02.828995 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:36:02.828995 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:36:02.828995 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:02.828995 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:02.828995 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 325/662] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:36:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:02.828995 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb11cd-e7f8-4e1d-87cf-3b68e722e8f7 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:02.851647 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dc6332-fbc7-492b-9aa7-6d7f6bc650a2 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:02.889413 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a07e8-b511-46cb-b45c-6e40a08dc223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:02.890034 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a07e8-b511-46cb-b45c-6e40a08dc223 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:02.890314 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a07e8-b511-46cb-b45c-6e40a08dc223 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:02.890589 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a07e8-b511-46cb-b45c-6e40a08dc223 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:02.906106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:02.941112 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:02.962279 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adb11cd-e7f8-4e1d-87cf-3b68e722e8f7 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:02.977576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a07e8-b511-46cb-b45c-6e40a08dc223 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:02.979009 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9a07e8-b511-46cb-b45c-6e40a08dc223 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:03.000928 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4dc856-d1c9-42e9-99b1-d083b96c8fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:03.001991 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4dc856-d1c9-42e9-99b1-d083b96c8fbe None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:03.002274 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4dc856-d1c9-42e9-99b1-d083b96c8fbe None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:03.002555 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4dc856-d1c9-42e9-99b1-d083b96c8fbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:03.079471 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4dc856-d1c9-42e9-99b1-d083b96c8fbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:03.081460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4dc856-d1c9-42e9-99b1-d083b96c8fbe None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:03.100179 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 338/663] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:36:02 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:36:03.101447 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd24402-3fa8-4f6b-a618-19a1207c46d2 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:03.101791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd24402-3fa8-4f6b-a618-19a1207c46d2 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:03.102014 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd24402-3fa8-4f6b-a618-19a1207c46d2 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:03.103457 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 339/664] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:03 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:03.108656 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd24402-3fa8-4f6b-a618-19a1207c46d2 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:03.157154 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:03.214752 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd24402-3fa8-4f6b-a618-19a1207c46d2 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:03.215493 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:03.237468 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 326/665] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:36:02 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:36:03.238490 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4dc856-d1c9-42e9-99b1-d083b96c8fbe None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:03.253026 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce69570-004b-4716-9c59-83660b29e758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:03.253649 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce69570-004b-4716-9c59-83660b29e758 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:03.253933 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce69570-004b-4716-9c59-83660b29e758 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:03.254211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce69570-004b-4716-9c59-83660b29e758 None admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:03.254988 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce69570-004b-4716-9c59-83660b29e758 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:03.255507 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce69570-004b-4716-9c59-83660b29e758 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:03.257001 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce69570-004b-4716-9c59-83660b29e758 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:03.268890 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 340/666] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:36:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:03.270113 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb43fa-45c1-4a7c-9da5-2d71f3947934 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:03.270379 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb43fa-45c1-4a7c-9da5-2d71f3947934 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:03.270660 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb43fa-45c1-4a7c-9da5-2d71f3947934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:03.289507 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-ddfb43fa-45c1-4a7c-9da5-2d71f3947934 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:03.290705 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-ddfb43fa-45c1-4a7c-9da5-2d71f3947934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:03.352087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4dc856-d1c9-42e9-99b1-d083b96c8fbe None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:03.361974 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 341/667] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:03 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:36:03.363273 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d842c3-27f7-42a0-b0cc-cf68c0b6e44b None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:03.363566 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d842c3-27f7-42a0-b0cc-cf68c0b6e44b None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:03.363846 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d842c3-27f7-42a0-b0cc-cf68c0b6e44b None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:03.365271 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 342/668] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:03 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:03.365790 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d842c3-27f7-42a0-b0cc-cf68c0b6e44b None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:03.391512 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0448e-069e-4202-ace4-d5082dd6a239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:03.392158 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0448e-069e-4202-ace4-d5082dd6a239 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:03.392438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0448e-069e-4202-ace4-d5082dd6a239 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:03.392717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0448e-069e-4202-ace4-d5082dd6a239 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:03.403072 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c0448e-069e-4202-ace4-d5082dd6a239 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:03.404546 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c0448e-069e-4202-ace4-d5082dd6a239 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:03.404674 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:03.420229 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-96c0448e-069e-4202-ace4-d5082dd6a239 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:03.420229 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:03.420229 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:03.420229 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:03.420229 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:03.420229 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:03.420229 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:03.420229 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:03.420229 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:03.420229 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:03.420229 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:03.420229 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:03.420229 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:03.420229 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:03.420229 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:03.422279 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:03.422279 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:03.422279 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:03.422279 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:03.422279 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:03.422279 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:03.422279 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:03.422279 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:03.422279 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:03.422279 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:03.422279 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:03.422279 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:03.422279 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:03.422279 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:03.422279 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:03.422279 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:03.422279 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:03.423412 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:03.423412 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:03.423412 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:03.423412 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:03.423412 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 327/669] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:03.423412 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0448e-069e-4202-ace4-d5082dd6a239 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:03.457837 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:03.488885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d842c3-27f7-42a0-b0cc-cf68c0b6e44b None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:03.524967 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2440c5-f0b5-4d5b-b7d1-32e87786112a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:03.525628 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2440c5-f0b5-4d5b-b7d1-32e87786112a None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:03.525909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2440c5-f0b5-4d5b-b7d1-32e87786112a None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:03.526191 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2440c5-f0b5-4d5b-b7d1-32e87786112a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:03.530470 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0448e-069e-4202-ace4-d5082dd6a239 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:03.537497 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2440c5-f0b5-4d5b-b7d1-32e87786112a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:03.539413 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2440c5-f0b5-4d5b-b7d1-32e87786112a None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:03.554434 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-9f2440c5-f0b5-4d5b-b7d1-32e87786112a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:03.554434 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:03.554434 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:03.554434 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:03.554434 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:03.554434 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:03.554434 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:03.554434 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:03.554434 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:03.554434 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:03.554434 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:03.554434 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:03.554434 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:03.554434 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:03.554434 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:03.555485 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:03.555485 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:03.555485 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:03.555485 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:03.555485 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:03.555485 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:03.555485 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:03.555485 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:03.555485 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:03.555485 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:03.555485 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:03.555485 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:03.555485 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:03.555485 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:03.555485 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:03.555485 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:03.555485 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:03.556426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:03.556426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:03.556426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:03.556426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:03.556426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 343/670] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:03.556901 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6c0f9-8627-4ee8-9557-d61fceb653a8 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:03.557109 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6c0f9-8627-4ee8-9557-d61fceb653a8 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:03.557339 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6c0f9-8627-4ee8-9557-d61fceb653a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:03.570500 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f75363-8f2a-443f-b9f6-23dea81568e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:03.571253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f75363-8f2a-443f-b9f6-23dea81568e4 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:03.571536 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f75363-8f2a-443f-b9f6-23dea81568e4 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:03.571813 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f75363-8f2a-443f-b9f6-23dea81568e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:03.572605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f75363-8f2a-443f-b9f6-23dea81568e4 None admin] RBAC: Adding query filter params (role.id=0f1c61dab22c4d97ae784468058036c7, scope.project.id=c1eae881e5714bf4bd223809e1a2d920, user.id=cc9ab3c9075b4a8385ef634dc0b1625e) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:03.572916 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f75363-8f2a-443f-b9f6-23dea81568e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:03.574363 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f75363-8f2a-443f-b9f6-23dea81568e4 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:03.575679 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-dfe6c0f9-8627-4ee8-9557-d61fceb653a8 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:03.577519 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-dfe6c0f9-8627-4ee8-9557-d61fceb653a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:03.600508 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 328/671] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:36:03 2022] GET /identity/v3/role_assignments?user.id=cc9ab3c9075b4a8385ef634dc0b1625e&scope.project.id=c1eae881e5714bf4bd223809e1a2d920&role.id=0f1c61dab22c4d97ae784468058036c7 => generated 610 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:03.601435 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f75363-8f2a-443f-b9f6-23dea81568e4 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:03.667176 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 344/672] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:03 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:36:03.667176 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6c0f9-8627-4ee8-9557-d61fceb653a8 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:03.684405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:03.732789 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:03.740870 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f75363-8f2a-443f-b9f6-23dea81568e4 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:03.778342 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27842fff-c2a6-482a-9f4e-584bfaf272b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:03.780719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27842fff-c2a6-482a-9f4e-584bfaf272b5 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:03.780719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27842fff-c2a6-482a-9f4e-584bfaf272b5 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:03.780719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27842fff-c2a6-482a-9f4e-584bfaf272b5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:03.784220 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6c0f9-8627-4ee8-9557-d61fceb653a8 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:03.793388 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27842fff-c2a6-482a-9f4e-584bfaf272b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:36:03.798942 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-27842fff-c2a6-482a-9f4e-584bfaf272b5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:36:03.798942 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:03.798942 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:03.798942 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:03.798942 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:03.798942 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:03.798942 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:03.798942 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:03.798942 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:03.798942 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:03.798942 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:03.798942 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:03.798942 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:36:03.798942 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:36:03.798942 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:36:03.798942 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:36:03.799972 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:36:03.799972 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:36:03.799972 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:36:03.799972 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:36:03.799972 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:03.799972 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:03.799972 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:03.799972 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:03.799972 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:03.799972 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:03.799972 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:03.799972 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:03.799972 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:03.799972 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:03.799972 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:03.799972 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:03.800834 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:03.800834 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:03.800834 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:03.800834 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:03.800834 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:36:03.800834 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:36:03.800834 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:36:03.800834 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:36:03.800834 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:36:03.800834 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:36:03.800834 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:36:03.800834 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:03.800834 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 329/673] 10.222.0.115 () {56 vars in 1107 bytes} [Tue Aug 2 10:36:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:03.801612 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27842fff-c2a6-482a-9f4e-584bfaf272b5 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:03.832099 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5206a2c-03a5-4042-a066-c3f21cacafc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:03.832780 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5206a2c-03a5-4042-a066-c3f21cacafc1 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:03.833060 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5206a2c-03a5-4042-a066-c3f21cacafc1 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:03.833324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5206a2c-03a5-4042-a066-c3f21cacafc1 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:03.834015 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5206a2c-03a5-4042-a066-c3f21cacafc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:03.834313 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5206a2c-03a5-4042-a066-c3f21cacafc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:03.835701 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5206a2c-03a5-4042-a066-c3f21cacafc1 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:03.849298 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 345/674] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:03.850126 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5206a2c-03a5-4042-a066-c3f21cacafc1 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:03.916939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27842fff-c2a6-482a-9f4e-584bfaf272b5 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:03.937914 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:03.953701 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7b2fd-bcb8-4294-a8bc-055e4de7850b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:03.954395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7b2fd-bcb8-4294-a8bc-055e4de7850b None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:03.954678 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7b2fd-bcb8-4294-a8bc-055e4de7850b None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:03.954949 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda7b2fd-bcb8-4294-a8bc-055e4de7850b None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:03.955919 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda7b2fd-bcb8-4294-a8bc-055e4de7850b None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:03.956216 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda7b2fd-bcb8-4294-a8bc-055e4de7850b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:03.957592 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda7b2fd-bcb8-4294-a8bc-055e4de7850b None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:03.966964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5206a2c-03a5-4042-a066-c3f21cacafc1 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:03.970117 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 330/675] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:03.970939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7b2fd-bcb8-4294-a8bc-055e4de7850b None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:03.993774 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:04.008484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea36ff-3d8d-48c9-9dd7-6f781c6c9f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:04.009121 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea36ff-3d8d-48c9-9dd7-6f781c6c9f1f None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:04.009398 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea36ff-3d8d-48c9-9dd7-6f781c6c9f1f None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:04.009821 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fea36ff-3d8d-48c9-9dd7-6f781c6c9f1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:04.020789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fea36ff-3d8d-48c9-9dd7-6f781c6c9f1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:36:04.026118 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-3fea36ff-3d8d-48c9-9dd7-6f781c6c9f1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:36:04.026118 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:04.026118 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:04.026118 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:04.026118 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:04.026118 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:04.026118 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:04.026118 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:04.026118 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:04.026118 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:04.026118 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:04.026118 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:04.026118 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:36:04.026118 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:36:04.026118 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:36:04.026118 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:36:04.027089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:36:04.027089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:36:04.027089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:36:04.027089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:36:04.027089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:04.027089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:04.027089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:04.027089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:04.027089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:04.027089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:04.027089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:04.027089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:04.027089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:04.027089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:04.027089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:04.027089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:04.027956 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:04.027956 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:04.027956 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:04.027956 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:04.027956 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:36:04.027956 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:36:04.027956 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:36:04.027956 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:36:04.027956 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:36:04.027956 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:36:04.027956 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:36:04.027956 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:04.027956 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 346/676] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:36:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:04.028653 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea36ff-3d8d-48c9-9dd7-6f781c6c9f1f None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:04.098325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda7b2fd-bcb8-4294-a8bc-055e4de7850b None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:04.133467 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fea36ff-3d8d-48c9-9dd7-6f781c6c9f1f None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:04.136355 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8591461-14f5-402b-9249-dc874daab734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:04.136962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8591461-14f5-402b-9249-dc874daab734 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:04.137236 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8591461-14f5-402b-9249-dc874daab734 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:04.137502 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8591461-14f5-402b-9249-dc874daab734 None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:04.138207 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8591461-14f5-402b-9249-dc874daab734 None admin] RBAC: Adding query filter params (name=service) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:04.138505 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8591461-14f5-402b-9249-dc874daab734 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:04.140428 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8591461-14f5-402b-9249-dc874daab734 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:04.152019 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 331/677] 10.222.0.115 () {56 vars in 1108 bytes} [Tue Aug 2 10:36:03 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:04.153622 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8591461-14f5-402b-9249-dc874daab734 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:04.175025 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f85543-1f8f-47a7-9614-3d45f4e755a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:04.175756 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f85543-1f8f-47a7-9614-3d45f4e755a1 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:04.176037 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f85543-1f8f-47a7-9614-3d45f4e755a1 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:04.176307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f85543-1f8f-47a7-9614-3d45f4e755a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:04.178309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:04.183819 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f85543-1f8f-47a7-9614-3d45f4e755a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:04.185237 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f85543-1f8f-47a7-9614-3d45f4e755a1 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:04.197035 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-97f85543-1f8f-47a7-9614-3d45f4e755a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:04.197035 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:04.197035 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:04.197035 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:04.197035 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:04.197035 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:04.197035 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:04.197035 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:04.197035 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:04.197035 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:04.197035 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:04.197035 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:04.197035 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:36:04.197035 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:36:04.197035 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:36:04.197940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:36:04.197940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:04.197940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:04.197940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:04.197940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:04.197940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:04.197940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:04.197940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:04.197940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:04.197940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:04.197940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:04.197940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:04.197940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:04.197940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:04.197940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:04.197940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:04.197940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:04.198819 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:36:04.198819 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:36:04.198819 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:36:04.198819 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:36:04.198819 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:36:04.198819 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:36:04.198819 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:04.198819 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:04.198819 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 347/678] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:36:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:04.199751 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f85543-1f8f-47a7-9614-3d45f4e755a1 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:04.247454 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:04.275807 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8591461-14f5-402b-9249-dc874daab734 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:04.311266 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f85543-1f8f-47a7-9614-3d45f4e755a1 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:04.313011 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d18c20f-13a8-48ac-b43b-fe5639763f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:04.313620 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18c20f-13a8-48ac-b43b-fe5639763f31 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:04.313894 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18c20f-13a8-48ac-b43b-fe5639763f31 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:04.314863 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18c20f-13a8-48ac-b43b-fe5639763f31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:04.321115 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d18c20f-13a8-48ac-b43b-fe5639763f31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:04.322541 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d18c20f-13a8-48ac-b43b-fe5639763f31 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:04.334880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-1d18c20f-13a8-48ac-b43b-fe5639763f31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:04.334880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:04.334880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:04.334880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:04.334880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:04.334880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:04.334880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:04.334880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:04.334880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:04.334880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:04.334880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:04.334880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:04.334880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:36:04.334880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:36:04.334880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:36:04.336006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:36:04.336006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:04.336006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:04.336006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:04.336006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:04.336006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:04.336006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:04.336006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:04.336006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:04.336006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:04.336006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:04.336006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:04.336006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:04.336006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:04.336006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:04.336006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:04.336006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:04.336948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:36:04.336948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:36:04.336948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:36:04.336948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:36:04.336948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:36:04.336948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:36:04.336948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:04.336948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:04.336948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 332/679] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:36:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:04.336948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d18c20f-13a8-48ac-b43b-fe5639763f31 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:04.355395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73643f48-42ae-4b4c-b3aa-051322b72905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:04.356054 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73643f48-42ae-4b4c-b3aa-051322b72905 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:04.356337 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73643f48-42ae-4b4c-b3aa-051322b72905 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:04.356638 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73643f48-42ae-4b4c-b3aa-051322b72905 None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:04.357388 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73643f48-42ae-4b4c-b3aa-051322b72905 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:04.357775 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73643f48-42ae-4b4c-b3aa-051322b72905 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:04.361247 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73643f48-42ae-4b4c-b3aa-051322b72905 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:04.369998 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 348/680] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:36:04 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:04.370813 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73643f48-42ae-4b4c-b3aa-051322b72905 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:04.440429 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:04.462721 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d18c20f-13a8-48ac-b43b-fe5639763f31 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:04.493011 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73643f48-42ae-4b4c-b3aa-051322b72905 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:04.505160 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f929e0-7953-44cc-8ccd-de39fbd71386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:04.505932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f929e0-7953-44cc-8ccd-de39fbd71386 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:04.506211 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f929e0-7953-44cc-8ccd-de39fbd71386 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:04.506481 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f929e0-7953-44cc-8ccd-de39fbd71386 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:04.513614 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:04.518496 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f929e0-7953-44cc-8ccd-de39fbd71386 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:04.519986 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f929e0-7953-44cc-8ccd-de39fbd71386 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:04.534240 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c324d51-1af2-4de2-a5f4-cb138f83d644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:04.534962 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c324d51-1af2-4de2-a5f4-cb138f83d644 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:04.535287 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c324d51-1af2-4de2-a5f4-cb138f83d644 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:04.535568 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c324d51-1af2-4de2-a5f4-cb138f83d644 None admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:04.536211 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-77f929e0-7953-44cc-8ccd-de39fbd71386 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:04.536211 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:04.536211 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:04.536211 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:04.536211 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:04.536211 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:04.536211 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:04.536211 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:04.536211 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:04.536211 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:04.536211 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:04.536211 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:04.536211 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:04.536211 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:04.536211 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:04.537080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:04.537080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:04.537080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:04.537080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:04.537080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:04.537080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:04.537080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:04.537080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:04.537080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:04.537080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:04.537080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:04.537080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:04.537080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:04.537080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:04.537080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:04.537080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:04.537080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:04.538080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:04.538080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:04.538080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:04.538080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:04.538799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c324d51-1af2-4de2-a5f4-cb138f83d644 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:04.538799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c324d51-1af2-4de2-a5f4-cb138f83d644 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:04.538799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c324d51-1af2-4de2-a5f4-cb138f83d644 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:04.542122 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 333/681] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:04.542954 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f929e0-7953-44cc-8ccd-de39fbd71386 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:04.551800 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 349/682] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:36:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:04.552980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c324d51-1af2-4de2-a5f4-cb138f83d644 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:04.661130 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f929e0-7953-44cc-8ccd-de39fbd71386 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:04.665029 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c324d51-1af2-4de2-a5f4-cb138f83d644 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:04.688877 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:04.699488 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cb5a4a-e760-4407-aa72-d0be3f6e98c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:04.700306 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb5a4a-e760-4407-aa72-d0be3f6e98c7 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:04.701269 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb5a4a-e760-4407-aa72-d0be3f6e98c7 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:04.701269 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cb5a4a-e760-4407-aa72-d0be3f6e98c7 None admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:04.702318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cb5a4a-e760-4407-aa72-d0be3f6e98c7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:04.702632 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cb5a4a-e760-4407-aa72-d0be3f6e98c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:04.704239 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cb5a4a-e760-4407-aa72-d0be3f6e98c7 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:04.710665 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e4132-15be-4307-9cbe-b862ce4c0cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:04.711510 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e4132-15be-4307-9cbe-b862ce4c0cb7 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:04.712022 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e4132-15be-4307-9cbe-b862ce4c0cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:04.712298 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415e4132-15be-4307-9cbe-b862ce4c0cb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:04.722552 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 334/683] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:36:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:04.722552 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cb5a4a-e760-4407-aa72-d0be3f6e98c7 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:04.724089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415e4132-15be-4307-9cbe-b862ce4c0cb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:04.726134 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415e4132-15be-4307-9cbe-b862ce4c0cb7 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:04.743869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-415e4132-15be-4307-9cbe-b862ce4c0cb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:04.743869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:04.743869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:04.743869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:04.743869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:04.743869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:04.743869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:04.743869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:04.743869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:04.743869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:04.743869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:04.743869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:04.743869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:04.743869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:04.743869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:04.744826 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:04.744826 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:04.744826 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:04.744826 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:04.744826 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:04.744826 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:04.744826 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:04.744826 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:04.744826 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:04.744826 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:04.744826 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:04.744826 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:04.744826 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:04.744826 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:04.744826 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:04.744826 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:04.744826 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:04.745715 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:04.745715 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:04.745715 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:04.745715 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:04.745715 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 350/684] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:04.746433 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e4132-15be-4307-9cbe-b862ce4c0cb7 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:04.784627 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:04.836835 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cb5a4a-e760-4407-aa72-d0be3f6e98c7 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:04.865260 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415e4132-15be-4307-9cbe-b862ce4c0cb7 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:04.874087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19518cbd-7986-42cf-b8c8-3d98f7af2fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:04.874793 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19518cbd-7986-42cf-b8c8-3d98f7af2fe4 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:04.875187 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19518cbd-7986-42cf-b8c8-3d98f7af2fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:04.875472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19518cbd-7986-42cf-b8c8-3d98f7af2fe4 None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:04.876188 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19518cbd-7986-42cf-b8c8-3d98f7af2fe4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:04.876546 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19518cbd-7986-42cf-b8c8-3d98f7af2fe4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:04.877913 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19518cbd-7986-42cf-b8c8-3d98f7af2fe4 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:04.890164 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 335/685] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:04 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:04.891241 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19518cbd-7986-42cf-b8c8-3d98f7af2fe4 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:04.908168 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8e14a-1f99-47fb-bea5-73c3dd13958a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:04.908795 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8e14a-1f99-47fb-bea5-73c3dd13958a None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:04.909080 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8e14a-1f99-47fb-bea5-73c3dd13958a None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:04.909360 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df8e14a-1f99-47fb-bea5-73c3dd13958a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:04.910155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df8e14a-1f99-47fb-bea5-73c3dd13958a None admin] RBAC: Adding query filter params (role.id=0f1c61dab22c4d97ae784468058036c7, scope.project.id=c1eae881e5714bf4bd223809e1a2d920, user.id=98dcb74152ec4375a586ea526e7f07c8) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:04.910469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df8e14a-1f99-47fb-bea5-73c3dd13958a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:04.911933 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df8e14a-1f99-47fb-bea5-73c3dd13958a None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:04.917869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 351/686] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:36:04 2022] GET /identity/v3/role_assignments?user.id=98dcb74152ec4375a586ea526e7f07c8&scope.project.id=c1eae881e5714bf4bd223809e1a2d920&role.id=0f1c61dab22c4d97ae784468058036c7 => generated 610 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:04.918644 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8e14a-1f99-47fb-bea5-73c3dd13958a None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:04.962224 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:05.023125 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19518cbd-7986-42cf-b8c8-3d98f7af2fe4 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:05.037811 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:05.044429 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df8e14a-1f99-47fb-bea5-73c3dd13958a None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:05.064852 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f68b5-ba11-4ac4-b1c8-e23aecc3d9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:05.065895 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f68b5-ba11-4ac4-b1c8-e23aecc3d9eb None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:05.066364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f68b5-ba11-4ac4-b1c8-e23aecc3d9eb None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:05.066955 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f68b5-ba11-4ac4-b1c8-e23aecc3d9eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:05.068252 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992f68b5-ba11-4ac4-b1c8-e23aecc3d9eb None admin] RBAC: Adding query filter params (role.id=0f1c61dab22c4d97ae784468058036c7, scope.project.id=c1eae881e5714bf4bd223809e1a2d920, user.id=e358fd7c1b3c43d7ac89fb3562678266) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:05.068935 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992f68b5-ba11-4ac4-b1c8-e23aecc3d9eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:05.070815 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992f68b5-ba11-4ac4-b1c8-e23aecc3d9eb None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:05.079877 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 336/687] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:36:04 2022] GET /identity/v3/role_assignments?user.id=e358fd7c1b3c43d7ac89fb3562678266&scope.project.id=c1eae881e5714bf4bd223809e1a2d920&role.id=0f1c61dab22c4d97ae784468058036c7 => generated 610 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:05.081505 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f68b5-ba11-4ac4-b1c8-e23aecc3d9eb None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:05.095959 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903ca61-219d-4c11-8cd4-759a8fa718d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:05.096614 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903ca61-219d-4c11-8cd4-759a8fa718d4 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:05.096905 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903ca61-219d-4c11-8cd4-759a8fa718d4 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:05.097185 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903ca61-219d-4c11-8cd4-759a8fa718d4 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:05.097913 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903ca61-219d-4c11-8cd4-759a8fa718d4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:05.098222 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903ca61-219d-4c11-8cd4-759a8fa718d4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:05.107137 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f903ca61-219d-4c11-8cd4-759a8fa718d4 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:05.114411 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 352/688] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:04 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:05.121279 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903ca61-219d-4c11-8cd4-759a8fa718d4 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:05.230618 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f68b5-ba11-4ac4-b1c8-e23aecc3d9eb None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:05.232645 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:05.243140 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903ca61-219d-4c11-8cd4-759a8fa718d4 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:05.277336 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7007ba-6e2d-4df1-9f41-6f5999000b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:05.278277 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7007ba-6e2d-4df1-9f41-6f5999000b24 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:05.278855 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7007ba-6e2d-4df1-9f41-6f5999000b24 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:05.279317 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7007ba-6e2d-4df1-9f41-6f5999000b24 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:05.287975 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e514c3d-c38c-4303-98d7-a0e3dcc62141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:05.288752 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e514c3d-c38c-4303-98d7-a0e3dcc62141 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:05.289040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e514c3d-c38c-4303-98d7-a0e3dcc62141 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:05.289319 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e514c3d-c38c-4303-98d7-a0e3dcc62141 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:05.291931 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7007ba-6e2d-4df1-9f41-6f5999000b24 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:05.293766 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7007ba-6e2d-4df1-9f41-6f5999000b24 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:05.301117 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e514c3d-c38c-4303-98d7-a0e3dcc62141 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:05.302875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e514c3d-c38c-4303-98d7-a0e3dcc62141 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:05.309547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-9c7007ba-6e2d-4df1-9f41-6f5999000b24 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:36:05.309547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:05.309547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:05.309547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:05.309547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:05.309547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:05.309547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:05.309547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:05.309547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:05.309547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:05.309547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:05.309547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:05.309547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:36:05.309547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:36:05.309547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:36:05.309547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:36:05.310572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:05.310572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:05.310572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:36:05.310572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:05.310572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:36:05.310572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:05.310572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:05.310572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:05.310572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:05.310572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:05.310572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:05.310572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:05.310572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:05.310572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:05.310572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:05.310572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:05.310572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:05.310572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:05.312214 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:05.312214 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:05.312214 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:36:05.312214 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:36:05.312214 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:36:05.312214 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:36:05.312214 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:36:05.312214 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:36:05.312214 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:36:05.312214 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:05.317550 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 337/689] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:36:05 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:05.318873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-1e514c3d-c38c-4303-98d7-a0e3dcc62141 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:36:05.318873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:05.318873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:05.318873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:05.318873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:05.318873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:05.318873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:05.318873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:05.318873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:05.318873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:05.318873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:05.318873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:05.318873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:36:05.318873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:36:05.318873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:36:05.318873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:36:05.321252 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:05.321252 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:05.321252 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:36:05.321252 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:05.321252 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:36:05.321252 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:05.321252 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:05.321252 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:05.321252 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:05.321252 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:05.321252 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:05.321252 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:05.321252 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:05.321252 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:05.321252 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:05.321252 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:05.321252 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:05.321252 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:05.324607 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:05.324607 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:05.324607 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:36:05.324607 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:36:05.324607 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:36:05.324607 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:36:05.324607 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:36:05.324607 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:36:05.324607 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:36:05.324607 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:05.325934 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7007ba-6e2d-4df1-9f41-6f5999000b24 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:05.326084 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 353/690] 10.222.0.115 () {56 vars in 1098 bytes} [Tue Aug 2 10:36:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:05.331109 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e514c3d-c38c-4303-98d7-a0e3dcc62141 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:05.378730 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:05.473331 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e514c3d-c38c-4303-98d7-a0e3dcc62141 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:05.476639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7007ba-6e2d-4df1-9f41-6f5999000b24 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:05.513402 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b0887-b53b-4917-8deb-bb5ef5bafa9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:05.513402 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b0887-b53b-4917-8deb-bb5ef5bafa9e None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:05.513698 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b0887-b53b-4917-8deb-bb5ef5bafa9e None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:05.513875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b0887-b53b-4917-8deb-bb5ef5bafa9e None admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:05.515093 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b0887-b53b-4917-8deb-bb5ef5bafa9e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:05.516411 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b0887-b53b-4917-8deb-bb5ef5bafa9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:05.517924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b0887-b53b-4917-8deb-bb5ef5bafa9e None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:05.532410 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118932a-1a39-4ba6-a665-217523f64068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:05.534735 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118932a-1a39-4ba6-a665-217523f64068 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:05.534735 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118932a-1a39-4ba6-a665-217523f64068 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:05.534735 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e118932a-1a39-4ba6-a665-217523f64068 None admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:05.534735 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e118932a-1a39-4ba6-a665-217523f64068 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:05.536103 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e118932a-1a39-4ba6-a665-217523f64068 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:05.538177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e118932a-1a39-4ba6-a665-217523f64068 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:05.539952 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 354/691] 10.222.0.115 () {56 vars in 1138 bytes} [Tue Aug 2 10:36:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:05.546320 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b0887-b53b-4917-8deb-bb5ef5bafa9e None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:05.562178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:05.571098 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 338/692] 10.222.0.115 () {56 vars in 1140 bytes} [Tue Aug 2 10:36:05 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:36:05.575753 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118932a-1a39-4ba6-a665-217523f64068 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:05.674732 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b0887-b53b-4917-8deb-bb5ef5bafa9e None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:05.703166 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:05.716950 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e118932a-1a39-4ba6-a665-217523f64068 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:05.719842 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb54c856-e833-4f60-bb59-67cfe739903a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:05.720658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54c856-e833-4f60-bb59-67cfe739903a None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:05.721946 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54c856-e833-4f60-bb59-67cfe739903a None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:05.722371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb54c856-e833-4f60-bb59-67cfe739903a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:05.776547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3ecbe7-bb1f-4756-811e-a3b0304f2533 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:05.777646 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ecbe7-bb1f-4756-811e-a3b0304f2533 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:05.778242 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ecbe7-bb1f-4756-811e-a3b0304f2533 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:05.778739 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3ecbe7-bb1f-4756-811e-a3b0304f2533 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:05.812277 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb54c856-e833-4f60-bb59-67cfe739903a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:05.814394 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb54c856-e833-4f60-bb59-67cfe739903a None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:05.829770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:05.858572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ecbe7-bb1f-4756-811e-a3b0304f2533 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:05.860722 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3ecbe7-bb1f-4756-811e-a3b0304f2533 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:05.953084 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 355/693] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:36:05 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:36:05.954712 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05465fb-d83e-4b24-83e4-f1cfed5747a4 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:05.955497 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05465fb-d83e-4b24-83e4-f1cfed5747a4 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:05.955981 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05465fb-d83e-4b24-83e4-f1cfed5747a4 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:05.961665 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 356/694] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:05 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:05.962793 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05465fb-d83e-4b24-83e4-f1cfed5747a4 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:05.985011 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:05.996065 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 339/695] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:36:05 2022] GET /identity/v3/auth/tokens => generated 800 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 02 10:36:05.997576 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5421d-6a7d-4183-bd17-b454842ecf17 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:05.998106 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5421d-6a7d-4183-bd17-b454842ecf17 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:05.998600 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5421d-6a7d-4183-bd17-b454842ecf17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:06.019823 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-f5e5421d-6a7d-4183-bd17-b454842ecf17 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:06.021434 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f5e5421d-6a7d-4183-bd17-b454842ecf17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:06.077231 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05465fb-d83e-4b24-83e4-f1cfed5747a4 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:06.089257 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:06.100014 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 340/696] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:05 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:36:06.102725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5421d-6a7d-4183-bd17-b454842ecf17 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:06.129285 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390bc25-67f9-48a5-bd32-a52a09c532f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:06.130046 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390bc25-67f9-48a5-bd32-a52a09c532f5 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:06.130404 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390bc25-67f9-48a5-bd32-a52a09c532f5 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:06.130758 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a390bc25-67f9-48a5-bd32-a52a09c532f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:06.152177 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390bc25-67f9-48a5-bd32-a52a09c532f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:06.153678 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a390bc25-67f9-48a5-bd32-a52a09c532f5 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:06.172230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-a390bc25-67f9-48a5-bd32-a52a09c532f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:06.172230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:06.172230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:06.172230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:06.172230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:06.172230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:06.172230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:06.172230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:06.172230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:06.172230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:06.172230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:06.172230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:06.172230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:06.172230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:06.172230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:06.173194 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:06.173194 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:06.173194 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:06.173194 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:06.173194 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:06.173194 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:06.173194 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:06.173194 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:06.173194 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:06.173194 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:06.173194 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:06.173194 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:06.173194 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:06.173194 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:06.173194 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:06.173194 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:06.173194 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:06.174136 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:06.174136 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:06.174136 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:06.174136 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:06.176477 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 357/697] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:06.177502 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390bc25-67f9-48a5-bd32-a52a09c532f5 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:06.244567 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5421d-6a7d-4183-bd17-b454842ecf17 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:06.278310 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:06.292300 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ebddc-a1d7-47da-abd6-17414166e80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:06.292956 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ebddc-a1d7-47da-abd6-17414166e80e None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:06.293238 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ebddc-a1d7-47da-abd6-17414166e80e None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:06.293514 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ebddc-a1d7-47da-abd6-17414166e80e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:06.306614 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ebddc-a1d7-47da-abd6-17414166e80e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:06.308144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ebddc-a1d7-47da-abd6-17414166e80e None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:06.325523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-551ebddc-a1d7-47da-abd6-17414166e80e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:06.325523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:06.325523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:06.325523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:06.325523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:06.325523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:06.325523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:06.325523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:06.325523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:06.325523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:06.325523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:06.325523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:06.325523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:06.325523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:06.325523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:06.327284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:06.327284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:06.327284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:06.327284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:06.327284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:06.327284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:06.327284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:06.327284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:06.327284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:06.327284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:06.327284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:06.327284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:06.327284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:06.327284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:06.327284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:06.327284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:06.327284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:06.328233 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:06.328233 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:06.328233 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:06.328233 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:06.328233 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 341/698] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:06.329067 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ebddc-a1d7-47da-abd6-17414166e80e None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:06.416571 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a390bc25-67f9-48a5-bd32-a52a09c532f5 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:06.453726 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91669a1b-9ab5-4ede-a443-806928a87502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:06.454780 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91669a1b-9ab5-4ede-a443-806928a87502 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:06.455558 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91669a1b-9ab5-4ede-a443-806928a87502 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:06.456042 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91669a1b-9ab5-4ede-a443-806928a87502 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:06.456905 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91669a1b-9ab5-4ede-a443-806928a87502 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:06.458539 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91669a1b-9ab5-4ede-a443-806928a87502 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:06.471531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-91669a1b-9ab5-4ede-a443-806928a87502 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 10:36:06.471531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:06.471531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:36:06.471531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:36:06.471531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:36:06.471531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:36:06.471531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:36:06.471531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:06.471531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 10:36:06.471531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:06.471531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:36:06.471531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:06.471531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:06.471531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:06.471531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:06.471531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:06.472570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:06.472570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:06.472570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:06.472570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:06.472570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:06.472570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:06.472570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:06.472570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:36:06.472570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:36:06.472570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:36:06.472570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:36:06.472570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:06.472570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:06.472570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:06.472570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:06.472570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:06.473423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:06.473423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:06.473423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:06.473423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:06.473423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:06.473423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:06.473423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:06.473423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:06.473423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:06.473423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:06.473423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:06.473423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:36:06.473423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:06.473423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 10:36:06.473423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:06.481168 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 358/699] 10.222.0.115 () {56 vars in 1134 bytes} [Tue Aug 2 10:36:06 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:06.482371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91669a1b-9ab5-4ede-a443-806928a87502 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:06.491602 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ebddc-a1d7-47da-abd6-17414166e80e None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:06.513720 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:06.558330 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3768bf8-eada-4c9c-a13b-e7fbc3be0037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:06.559324 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3768bf8-eada-4c9c-a13b-e7fbc3be0037 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:06.559767 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3768bf8-eada-4c9c-a13b-e7fbc3be0037 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:06.560793 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3768bf8-eada-4c9c-a13b-e7fbc3be0037 None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:06.561525 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3768bf8-eada-4c9c-a13b-e7fbc3be0037 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:06.562056 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3768bf8-eada-4c9c-a13b-e7fbc3be0037 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:06.564761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3768bf8-eada-4c9c-a13b-e7fbc3be0037 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:06.589723 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 342/700] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:06.591190 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3768bf8-eada-4c9c-a13b-e7fbc3be0037 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:06.599557 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91669a1b-9ab5-4ede-a443-806928a87502 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:06.636170 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe441f21-10c8-41df-b06e-b46ce5a3546f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:06.637157 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe441f21-10c8-41df-b06e-b46ce5a3546f None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:06.637669 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe441f21-10c8-41df-b06e-b46ce5a3546f None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:06.638147 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe441f21-10c8-41df-b06e-b46ce5a3546f None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:06.639655 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe441f21-10c8-41df-b06e-b46ce5a3546f None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:06.640172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe441f21-10c8-41df-b06e-b46ce5a3546f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:06.641740 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe441f21-10c8-41df-b06e-b46ce5a3546f None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:06.655396 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 359/701] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:06.656554 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe441f21-10c8-41df-b06e-b46ce5a3546f None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:06.659566 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:06.775903 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3768bf8-eada-4c9c-a13b-e7fbc3be0037 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:06.794227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:06.814348 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d345c-ac08-41b0-a5ec-ee24dac3d9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:06.814582 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe441f21-10c8-41df-b06e-b46ce5a3546f None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:06.815560 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d345c-ac08-41b0-a5ec-ee24dac3d9d6 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:06.816047 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d345c-ac08-41b0-a5ec-ee24dac3d9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:06.816521 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36d345c-ac08-41b0-a5ec-ee24dac3d9d6 None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:06.817395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36d345c-ac08-41b0-a5ec-ee24dac3d9d6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:06.817952 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36d345c-ac08-41b0-a5ec-ee24dac3d9d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:06.819594 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36d345c-ac08-41b0-a5ec-ee24dac3d9d6 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:06.824736 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 343/702] 10.222.0.115 () {56 vars in 1129 bytes} [Tue Aug 2 10:36:06 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:06.830664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d345c-ac08-41b0-a5ec-ee24dac3d9d6 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:06.857893 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef419952-9f02-47bd-b203-aeca420828a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:06.858466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef419952-9f02-47bd-b203-aeca420828a9 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:06.858745 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef419952-9f02-47bd-b203-aeca420828a9 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:06.859017 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef419952-9f02-47bd-b203-aeca420828a9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:06.868802 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef419952-9f02-47bd-b203-aeca420828a9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:06.870501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef419952-9f02-47bd-b203-aeca420828a9 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:06.885927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-ef419952-9f02-47bd-b203-aeca420828a9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:06.885927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:06.885927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:06.885927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:06.885927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:06.885927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:06.885927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:06.885927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:06.885927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:06.885927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:06.885927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:06.885927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:06.885927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:36:06.885927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:36:06.885927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:36:06.886885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:36:06.886885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:06.886885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:06.886885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:06.886885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:06.886885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:06.886885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:06.886885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:06.886885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:06.886885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:06.886885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:06.886885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:06.886885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:06.886885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:06.886885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:06.886885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:06.886885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:06.887910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:36:06.887910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:36:06.887910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:36:06.887910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:36:06.887910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:36:06.887910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:36:06.887910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:06.887910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:06.887910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 360/703] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:36:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:06.888454 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef419952-9f02-47bd-b203-aeca420828a9 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:06.952947 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:06.958089 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36d345c-ac08-41b0-a5ec-ee24dac3d9d6 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:06.995897 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1dfe86-8435-41d6-af76-d634a01e4cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:06.996888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1dfe86-8435-41d6-af76-d634a01e4cef None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:06.997400 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1dfe86-8435-41d6-af76-d634a01e4cef None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:06.997896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1dfe86-8435-41d6-af76-d634a01e4cef None admin] PATH_INFO: `/v3/projects/service` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:07.005648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1dfe86-8435-41d6-af76-d634a01e4cef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:07.007434 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1dfe86-8435-41d6-af76-d634a01e4cef None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:07.019768 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-5d1dfe86-8435-41d6-af76-d634a01e4cef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:07.019768 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:07.019768 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:07.019768 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:07.019768 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:07.019768 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:07.019768 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:07.019768 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:07.019768 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:07.019768 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:07.019768 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:07.019768 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:07.019768 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:36:07.019768 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:36:07.019768 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:36:07.021338 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:36:07.021338 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:07.021338 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:07.021338 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:07.021338 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:07.021338 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:07.021338 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:07.021338 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:07.021338 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:07.021338 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:07.021338 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:07.021338 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:07.021338 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:07.021338 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:07.021338 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:07.021338 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:07.021338 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:07.023172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:36:07.023172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:36:07.023172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:36:07.023172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:36:07.023172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:36:07.023172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:36:07.023172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:07.023172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:07.025658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 344/704] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:36:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:07.025960 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef419952-9f02-47bd-b203-aeca420828a9 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:07.027403 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1dfe86-8435-41d6-af76-d634a01e4cef None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:07.075000 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99575f00-311a-41bb-8269-d8268a8f7eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:07.076065 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99575f00-311a-41bb-8269-d8268a8f7eb2 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:07.077486 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99575f00-311a-41bb-8269-d8268a8f7eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:07.077997 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99575f00-311a-41bb-8269-d8268a8f7eb2 None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:07.078967 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99575f00-311a-41bb-8269-d8268a8f7eb2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:07.079504 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99575f00-311a-41bb-8269-d8268a8f7eb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:07.081341 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99575f00-311a-41bb-8269-d8268a8f7eb2 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:07.093571 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 361/705] 10.222.0.115 () {56 vars in 1129 bytes} [Tue Aug 2 10:36:06 2022] GET /identity/v3/services?type=block-storage => generated 131 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:07.094966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99575f00-311a-41bb-8269-d8268a8f7eb2 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:07.119610 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:07.171615 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1dfe86-8435-41d6-af76-d634a01e4cef None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:07.251793 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b4c4b-29ad-4eb0-bf7f-8e1812f9ebbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:07.252485 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b4c4b-29ad-4eb0-bf7f-8e1812f9ebbb None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:07.252778 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b4c4b-29ad-4eb0-bf7f-8e1812f9ebbb None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:07.253048 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b4c4b-29ad-4eb0-bf7f-8e1812f9ebbb None admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:07.253815 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0b4c4b-29ad-4eb0-bf7f-8e1812f9ebbb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:07.254123 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0b4c4b-29ad-4eb0-bf7f-8e1812f9ebbb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:07.255603 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0b4c4b-29ad-4eb0-bf7f-8e1812f9ebbb None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:07.278748 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 345/706] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:36:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:07.288140 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b4c4b-29ad-4eb0-bf7f-8e1812f9ebbb None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:07.320958 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99575f00-311a-41bb-8269-d8268a8f7eb2 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:07.321211 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:07.371029 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e2b4be-5a5b-486a-a20b-626c8ab9008c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:07.371745 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2b4be-5a5b-486a-a20b-626c8ab9008c None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:07.372419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2b4be-5a5b-486a-a20b-626c8ab9008c None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:07.372572 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e2b4be-5a5b-486a-a20b-626c8ab9008c None admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:07.373872 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e2b4be-5a5b-486a-a20b-626c8ab9008c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:07.374380 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e2b4be-5a5b-486a-a20b-626c8ab9008c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:07.375837 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e2b4be-5a5b-486a-a20b-626c8ab9008c None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:07.389998 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 362/707] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:36:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:07.396025 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e2b4be-5a5b-486a-a20b-626c8ab9008c None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:07.422640 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b4c4b-29ad-4eb0-bf7f-8e1812f9ebbb None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:07.460023 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e8ec8a-c7f3-4c14-a2e3-622a53dd41b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:07.461627 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8ec8a-c7f3-4c14-a2e3-622a53dd41b4 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:07.462134 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8ec8a-c7f3-4c14-a2e3-622a53dd41b4 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:07.462613 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e8ec8a-c7f3-4c14-a2e3-622a53dd41b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:07.464055 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8ec8a-c7f3-4c14-a2e3-622a53dd41b4 None admin] RBAC: Adding query filter params (role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac, scope.project.id=c1eae881e5714bf4bd223809e1a2d920, user.id=d0478bd540f446ec810e2091c46ac49c) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:07.464804 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8ec8a-c7f3-4c14-a2e3-622a53dd41b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:07.467004 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e8ec8a-c7f3-4c14-a2e3-622a53dd41b4 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:07.474031 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 346/708] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:36:07 2022] GET /identity/v3/role_assignments?user.id=d0478bd540f446ec810e2091c46ac49c&scope.project.id=c1eae881e5714bf4bd223809e1a2d920&role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 260 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:07.547622 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:07.578036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb11f5-db65-4217-b433-98907838edda None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:07.579086 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb11f5-db65-4217-b433-98907838edda None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:07.579680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb11f5-db65-4217-b433-98907838edda None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:07.581880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 347/709] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:07 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:07.587980 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c977d14-673c-4f70-8cd1-86673366b1ce None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:07.588592 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c977d14-673c-4f70-8cd1-86673366b1ce None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:07.589102 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c977d14-673c-4f70-8cd1-86673366b1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:07.589628 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e2b4be-5a5b-486a-a20b-626c8ab9008c None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:07.616073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-0c977d14-673c-4f70-8cd1-86673366b1ce None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:07.617734 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-0c977d14-673c-4f70-8cd1-86673366b1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:07.653253 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce33a51-1287-41bb-a64a-141008ace772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:07.653839 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce33a51-1287-41bb-a64a-141008ace772 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:07.654124 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce33a51-1287-41bb-a64a-141008ace772 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:07.654420 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce33a51-1287-41bb-a64a-141008ace772 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:07.655999 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce33a51-1287-41bb-a64a-141008ace772 None admin] RBAC: Adding query filter params (role.id=0f1c61dab22c4d97ae784468058036c7, scope.project.id=c1eae881e5714bf4bd223809e1a2d920, user.id=0daa49b5a35f4a468bb8d0939e7ec20c) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:07.656312 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce33a51-1287-41bb-a64a-141008ace772 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:07.657760 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce33a51-1287-41bb-a64a-141008ace772 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:07.666485 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 363/710] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:36:07 2022] GET /identity/v3/role_assignments?user.id=0daa49b5a35f4a468bb8d0939e7ec20c&scope.project.id=c1eae881e5714bf4bd223809e1a2d920&role.id=0f1c61dab22c4d97ae784468058036c7 => generated 610 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:07.706747 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 348/711] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:07 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:36:07.727755 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce33a51-1287-41bb-a64a-141008ace772 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:07.877958 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce33a51-1287-41bb-a64a-141008ace772 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:07.926574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e62e6c6-7bb7-4dd5-88f0-d95da5492dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:07.927178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62e6c6-7bb7-4dd5-88f0-d95da5492dbe None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:07.927462 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62e6c6-7bb7-4dd5-88f0-d95da5492dbe None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:07.927852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e62e6c6-7bb7-4dd5-88f0-d95da5492dbe None admin] PATH_INFO: `/v3/services/network` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:07.928488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e62e6c6-7bb7-4dd5-88f0-d95da5492dbe None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:07.929860 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e62e6c6-7bb7-4dd5-88f0-d95da5492dbe None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:07.933185 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:07.945087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-0e62e6c6-7bb7-4dd5-88f0-d95da5492dbe None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 10:36:07.945087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:07.945087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:36:07.945087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:36:07.945087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:36:07.945087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:36:07.945087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:36:07.945087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:07.945087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 10:36:07.945087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:07.945087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:36:07.945087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:07.945087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:07.945087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:07.945087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:07.945087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:07.945087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:07.946303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:07.946303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:07.946303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:07.946303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:07.946303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:07.946303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:07.946303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:36:07.946303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:36:07.946303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:36:07.946303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:36:07.946303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:07.946303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:07.946303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:07.946303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:07.946303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:07.946303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:07.947199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:07.947199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:07.947199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:07.947199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:07.947199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:07.947199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:07.947199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:07.947199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:07.947199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:07.947199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:07.947199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:36:07.947199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:07.947199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 10:36:07.947199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:07.948041 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 364/712] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:36:07 2022] GET /identity/v3/services/network => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:07.955771 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c977d14-673c-4f70-8cd1-86673366b1ce None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:07.966316 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d528e-01af-4aa5-8f5a-9e5f7d50c68a None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:07.966627 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d528e-01af-4aa5-8f5a-9e5f7d50c68a None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:07.967595 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d528e-01af-4aa5-8f5a-9e5f7d50c68a None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:07.970339 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 365/713] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:07 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:36:07.976746 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861f148-8004-4664-8ac6-e1e3148b5472 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:07.977577 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861f148-8004-4664-8ac6-e1e3148b5472 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:07.977932 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861f148-8004-4664-8ac6-e1e3148b5472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:07.983477 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:08.000483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-a861f148-8004-4664-8ac6-e1e3148b5472 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:08.002966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-a861f148-8004-4664-8ac6-e1e3148b5472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:08.085965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c977d14-673c-4f70-8cd1-86673366b1ce None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:08.091839 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 366/714] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:07 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:36:08.124154 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a861f148-8004-4664-8ac6-e1e3148b5472 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:08.127391 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4518c4c-feba-4236-89fa-8143421ff704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:08.128317 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4518c4c-feba-4236-89fa-8143421ff704 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:08.128779 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4518c4c-feba-4236-89fa-8143421ff704 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:08.129232 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4518c4c-feba-4236-89fa-8143421ff704 None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:08.130216 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4518c4c-feba-4236-89fa-8143421ff704 None admin] RBAC: Adding query filter params (name=network) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:08.130746 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4518c4c-feba-4236-89fa-8143421ff704 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:08.132366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4518c4c-feba-4236-89fa-8143421ff704 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:08.138199 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 349/715] 10.222.0.115 () {56 vars in 1117 bytes} [Tue Aug 2 10:36:07 2022] GET /identity/v3/services?name=network => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:08.144602 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4518c4c-feba-4236-89fa-8143421ff704 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:08.237873 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:08.286867 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4518c4c-feba-4236-89fa-8143421ff704 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:08.350735 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b789ee2-0859-4375-9cae-c4024377fe3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:08.351571 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b789ee2-0859-4375-9cae-c4024377fe3c None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:08.352290 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b789ee2-0859-4375-9cae-c4024377fe3c None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:08.353182 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b789ee2-0859-4375-9cae-c4024377fe3c None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:08.355314 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b789ee2-0859-4375-9cae-c4024377fe3c None admin] RBAC: Adding query filter params (type=network) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:08.355471 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a861f148-8004-4664-8ac6-e1e3148b5472 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:08.355957 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b789ee2-0859-4375-9cae-c4024377fe3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:08.358742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b789ee2-0859-4375-9cae-c4024377fe3c None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:08.368656 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 350/716] 10.222.0.115 () {56 vars in 1117 bytes} [Tue Aug 2 10:36:08 2022] GET /identity/v3/services?type=network => generated 125 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:08.409804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:08.428013 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6433a-d73c-43fb-a13e-866e243a46a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:08.428951 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6433a-d73c-43fb-a13e-866e243a46a3 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:08.429236 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6433a-d73c-43fb-a13e-866e243a46a3 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:08.429623 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6433a-d73c-43fb-a13e-866e243a46a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:08.454833 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e6433a-d73c-43fb-a13e-866e243a46a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:08.456696 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e6433a-d73c-43fb-a13e-866e243a46a3 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:08.480976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-29e6433a-d73c-43fb-a13e-866e243a46a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:08.480976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:08.480976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:08.480976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:08.480976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:08.480976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:08.480976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:08.480976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:08.480976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:08.480976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:08.480976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:08.480976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:08.480976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:08.480976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:08.480976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:08.481976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:08.481976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:08.481976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:08.481976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:08.481976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:08.481976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:08.481976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:08.481976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:08.481976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:08.481976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:08.481976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:08.481976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:08.481976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:08.481976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:08.481976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:08.481976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:08.481976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:08.483826 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:08.483826 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:08.483826 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:08.483826 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:08.484944 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 367/717] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:08.490300 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b789ee2-0859-4375-9cae-c4024377fe3c None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:08.666356 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b789ee2-0859-4375-9cae-c4024377fe3c None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:08.706633 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3067e9-fa3b-4a1d-88ab-1ec45fa1543a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:08.707652 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3067e9-fa3b-4a1d-88ab-1ec45fa1543a None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:08.708163 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3067e9-fa3b-4a1d-88ab-1ec45fa1543a None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:08.708639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3067e9-fa3b-4a1d-88ab-1ec45fa1543a None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:08.709567 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3067e9-fa3b-4a1d-88ab-1ec45fa1543a None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:08.710073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3067e9-fa3b-4a1d-88ab-1ec45fa1543a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:08.711678 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3067e9-fa3b-4a1d-88ab-1ec45fa1543a None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:08.722850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:08.727622 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 351/718] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:08 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:36:08.733527 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6433a-d73c-43fb-a13e-866e243a46a3 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:08.899862 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6433a-d73c-43fb-a13e-866e243a46a3 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:08.950805 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d85bad-41a1-4d30-8a20-3051d087f24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:08.951975 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d85bad-41a1-4d30-8a20-3051d087f24a None admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:08.952943 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d85bad-41a1-4d30-8a20-3051d087f24a None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:08.953220 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d85bad-41a1-4d30-8a20-3051d087f24a None admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:08.953915 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d85bad-41a1-4d30-8a20-3051d087f24a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:08.955639 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d85bad-41a1-4d30-8a20-3051d087f24a None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:09.085178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 368/719] 10.222.0.115 () {60 vars in 1137 bytes} [Tue Aug 2 10:36:08 2022] POST /identity/v3/users => generated 255 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:36:09.687861 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9635314-e2c2-4440-a715-ff8f312d1f35 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:09.688891 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9635314-e2c2-4440-a715-ff8f312d1f35 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:09.689377 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9635314-e2c2-4440-a715-ff8f312d1f35 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:09.691268 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 352/720] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:09 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:09.696451 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e373a-c8f6-4f3a-9d2c-e60dccebe0aa None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:09.697338 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e373a-c8f6-4f3a-9d2c-e60dccebe0aa None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:09.697866 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4e373a-c8f6-4f3a-9d2c-e60dccebe0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:09.719341 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-9c4e373a-c8f6-4f3a-9d2c-e60dccebe0aa None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:09.720873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-9c4e373a-c8f6-4f3a-9d2c-e60dccebe0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:09.755315 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:09.809726 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 369/721] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:09 2022] POST /identity/v3/auth/tokens => generated 800 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:36:09.846324 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9635314-e2c2-4440-a715-ff8f312d1f35 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:09.980849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9635314-e2c2-4440-a715-ff8f312d1f35 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:10.036609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d08f35a-9e87-4f11-9b60-dc5c8506f3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:10.037318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08f35a-9e87-4f11-9b60-dc5c8506f3c5 None admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:10.037611 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08f35a-9e87-4f11-9b60-dc5c8506f3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:10.037891 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d08f35a-9e87-4f11-9b60-dc5c8506f3c5 None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:10.038603 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d08f35a-9e87-4f11-9b60-dc5c8506f3c5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:10.040002 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d08f35a-9e87-4f11-9b60-dc5c8506f3c5 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:10.153804 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 353/722] 10.222.0.115 () {60 vars in 1146 bytes} [Tue Aug 2 10:36:09 2022] POST /identity/v3/services => generated 252 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:36:10.241998 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48054d4-f2b2-4c28-a782-30df23a9c649 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:10.242621 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48054d4-f2b2-4c28-a782-30df23a9c649 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:10.243159 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48054d4-f2b2-4c28-a782-30df23a9c649 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:10.245169 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 370/723] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:10 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:10.252966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732f4eb-babd-4c69-a446-f6cad99b9f6e None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:10.253300 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732f4eb-babd-4c69-a446-f6cad99b9f6e None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:10.253545 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d732f4eb-babd-4c69-a446-f6cad99b9f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:10.281401 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-d732f4eb-babd-4c69-a446-f6cad99b9f6e None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:10.282776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-d732f4eb-babd-4c69-a446-f6cad99b9f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:10.369492 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:10.400184 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 354/724] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:10 2022] POST /identity/v3/auth/tokens => generated 904 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:36:10.424697 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48054d4-f2b2-4c28-a782-30df23a9c649 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:10.548614 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48054d4-f2b2-4c28-a782-30df23a9c649 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:10.576092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efccfe0a-7268-4e02-841d-f55c803a2771 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:10.576539 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efccfe0a-7268-4e02-841d-f55c803a2771 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:10.576998 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efccfe0a-7268-4e02-841d-f55c803a2771 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:10.579472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 355/725] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:10 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:10.588478 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b98ee7-851c-4134-9226-c426ec5245fe None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:10.589222 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b98ee7-851c-4134-9226-c426ec5245fe None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:10.589747 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b98ee7-851c-4134-9226-c426ec5245fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:10.595299 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de95991-ee1c-4b63-ad31-0de609d2aeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:10.595952 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de95991-ee1c-4b63-ad31-0de609d2aeda None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:10.596233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de95991-ee1c-4b63-ad31-0de609d2aeda None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:10.596509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de95991-ee1c-4b63-ad31-0de609d2aeda None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:10.608718 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de95991-ee1c-4b63-ad31-0de609d2aeda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:36:10.613929 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-9de95991-ee1c-4b63-ad31-0de609d2aeda None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:36:10.613929 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:10.613929 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:10.613929 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:10.613929 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:10.613929 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:10.613929 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:10.613929 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:10.613929 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:10.613929 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:10.613929 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:10.613929 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:10.613929 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:36:10.613929 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:36:10.613929 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:36:10.613929 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:36:10.614910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:36:10.614910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:36:10.614910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:36:10.614910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:36:10.614910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:10.614910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:10.614910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:10.614910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:10.614910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:10.614910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:10.614910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:10.614910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:10.614910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:10.614910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:10.614910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:10.614910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:10.615850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:10.615850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:10.615850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:10.615850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:10.615850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:36:10.615850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:36:10.615850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:36:10.615850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:36:10.615850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:36:10.615850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:36:10.615850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:36:10.615850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:10.615850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 371/726] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:36:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:10.618817 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-18b98ee7-851c-4134-9226-c426ec5245fe None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:10.620061 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-18b98ee7-851c-4134-9226-c426ec5245fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:10.623112 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de95991-ee1c-4b63-ad31-0de609d2aeda None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:10.668898 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:10.735848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de95991-ee1c-4b63-ad31-0de609d2aeda None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:10.782186 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81938a46-0029-4404-a202-cfdf53345b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:10.782842 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81938a46-0029-4404-a202-cfdf53345b1b None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:10.783150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81938a46-0029-4404-a202-cfdf53345b1b None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:10.783435 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81938a46-0029-4404-a202-cfdf53345b1b None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:10.784179 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81938a46-0029-4404-a202-cfdf53345b1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:10.784484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81938a46-0029-4404-a202-cfdf53345b1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:10.785891 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81938a46-0029-4404-a202-cfdf53345b1b None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:10.807459 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 372/727] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:36:10 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:10.817319 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81938a46-0029-4404-a202-cfdf53345b1b None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:11.009101 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:11.014825 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 356/728] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:10 2022] POST /identity/v3/auth/tokens => generated 904 bytes in 427 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 02 10:36:11.020962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad781ca-04a4-45db-b709-76a4507dfd5b None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:11.020962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad781ca-04a4-45db-b709-76a4507dfd5b None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:11.021318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad781ca-04a4-45db-b709-76a4507dfd5b None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:11.024328 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 357/729] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:11 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:11.028710 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0dd64-7d2b-44c5-a723-01d90a14ade5 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:11.028999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0dd64-7d2b-44c5-a723-01d90a14ade5 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:11.029500 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf0dd64-7d2b-44c5-a723-01d90a14ade5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:11.040488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81938a46-0029-4404-a202-cfdf53345b1b None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:11.064213 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-3bf0dd64-7d2b-44c5-a723-01d90a14ade5 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:11.065378 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-3bf0dd64-7d2b-44c5-a723-01d90a14ade5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:11.098336 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713a6a2-9a7e-401d-ab9a-8d0b7bd4cc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:11.099824 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713a6a2-9a7e-401d-ab9a-8d0b7bd4cc32 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:11.100671 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713a6a2-9a7e-401d-ab9a-8d0b7bd4cc32 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:11.101319 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8713a6a2-9a7e-401d-ab9a-8d0b7bd4cc32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:11.119024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713a6a2-9a7e-401d-ab9a-8d0b7bd4cc32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:11.121675 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8713a6a2-9a7e-401d-ab9a-8d0b7bd4cc32 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:11.137574 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 358/730] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:11 2022] POST /identity/v3/auth/tokens => generated 904 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 02 10:36:11.138508 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0dd64-7d2b-44c5-a723-01d90a14ade5 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:11.146484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-8713a6a2-9a7e-401d-ab9a-8d0b7bd4cc32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:11.146484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:11.146484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:11.146484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:11.146484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:11.146484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:11.146484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:11.146484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:11.146484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:11.146484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:11.146484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:11.146484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:11.146484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:11.146484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:11.146484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:11.148233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:11.148233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:11.148233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:11.148233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:11.148233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:11.148233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:11.148233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:11.148233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:11.148233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:11.148233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:11.148233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:11.148233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:11.148233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:11.148233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:11.148233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:11.148233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:11.148233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:11.149776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:11.149776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:11.149776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:11.149776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:11.152354 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 373/731] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:36:11.160354 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713a6a2-9a7e-401d-ab9a-8d0b7bd4cc32 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:11.216715 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:11.259222 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf0dd64-7d2b-44c5-a723-01d90a14ade5 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:11.298852 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178fc97-12f2-4dad-90b3-b328a1f039d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:11.299627 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178fc97-12f2-4dad-90b3-b328a1f039d3 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:11.299979 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178fc97-12f2-4dad-90b3-b328a1f039d3 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:11.300326 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d178fc97-12f2-4dad-90b3-b328a1f039d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:11.314609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d178fc97-12f2-4dad-90b3-b328a1f039d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:36:11.320747 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-d178fc97-12f2-4dad-90b3-b328a1f039d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:36:11.320747 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:11.320747 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:11.320747 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:11.320747 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:11.320747 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:11.320747 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:11.320747 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:11.320747 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:11.320747 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:11.320747 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:11.320747 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:11.320747 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:36:11.320747 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:36:11.320747 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:36:11.320747 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:36:11.322517 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:36:11.322517 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:36:11.322517 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:36:11.322517 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:36:11.322517 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:11.322517 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:11.322517 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:11.322517 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:11.322517 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:11.322517 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:11.322517 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:11.322517 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:11.322517 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:11.322517 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:11.322517 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:11.322517 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:11.324036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:11.324036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:11.324036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:11.324036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:11.324036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:36:11.324036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:36:11.324036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:36:11.324036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:36:11.324036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:36:11.324036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:36:11.324036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:36:11.324036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:11.324036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 359/732] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:36:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:11.325202 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178fc97-12f2-4dad-90b3-b328a1f039d3 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:11.351790 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8713a6a2-9a7e-401d-ab9a-8d0b7bd4cc32 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:11.398228 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c50ad-82ef-4a1a-8489-dbd6d84fe7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:11.399440 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c50ad-82ef-4a1a-8489-dbd6d84fe7db None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:11.399964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c50ad-82ef-4a1a-8489-dbd6d84fe7db None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:11.400512 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c50ad-82ef-4a1a-8489-dbd6d84fe7db None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:11.401820 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068c50ad-82ef-4a1a-8489-dbd6d84fe7db None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:11.402603 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068c50ad-82ef-4a1a-8489-dbd6d84fe7db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:11.407801 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068c50ad-82ef-4a1a-8489-dbd6d84fe7db None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:11.419945 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:11.426312 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 374/733] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:11 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:36:11.427540 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c50ad-82ef-4a1a-8489-dbd6d84fe7db None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:11.469193 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d178fc97-12f2-4dad-90b3-b328a1f039d3 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:11.480113 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:11.520394 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aea521-e13d-4df0-ab44-ea6717a822fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:11.522077 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aea521-e13d-4df0-ab44-ea6717a822fb None admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:11.522567 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aea521-e13d-4df0-ab44-ea6717a822fb None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:11.523331 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aea521-e13d-4df0-ab44-ea6717a822fb None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:11.524456 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aea521-e13d-4df0-ab44-ea6717a822fb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:11.526759 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aea521-e13d-4df0-ab44-ea6717a822fb None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:11.586416 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c50ad-82ef-4a1a-8489-dbd6d84fe7db None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:11.619733 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 360/734] 10.222.0.115 () {60 vars in 1146 bytes} [Tue Aug 2 10:36:11 2022] POST /identity/v3/services => generated 241 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:36:11.619733 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aea521-e13d-4df0-ab44-ea6717a822fb None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:11.696171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3eeef0-d9f7-4daf-bd92-37ece2c56d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:11.696697 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3eeef0-d9f7-4daf-bd92-37ece2c56d4d None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:11.697027 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3eeef0-d9f7-4daf-bd92-37ece2c56d4d None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:11.697321 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3eeef0-d9f7-4daf-bd92-37ece2c56d4d None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:11.698310 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3eeef0-d9f7-4daf-bd92-37ece2c56d4d None admin] RBAC: Adding query filter params (name=admin) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:11.698555 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3eeef0-d9f7-4daf-bd92-37ece2c56d4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:11.700037 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3eeef0-d9f7-4daf-bd92-37ece2c56d4d None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:11.730623 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 375/735] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:36:11 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:11.740142 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3eeef0-d9f7-4daf-bd92-37ece2c56d4d None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:11.769670 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aea521-e13d-4df0-ab44-ea6717a822fb None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:11.824228 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da607312-8bee-47c5-acb4-d4a9257b3312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:11.825261 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da607312-8bee-47c5-acb4-d4a9257b3312 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:11.825757 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da607312-8bee-47c5-acb4-d4a9257b3312 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:11.826786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da607312-8bee-47c5-acb4-d4a9257b3312 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:11.840469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da607312-8bee-47c5-acb4-d4a9257b3312 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:11.843126 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da607312-8bee-47c5-acb4-d4a9257b3312 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:11.859557 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-da607312-8bee-47c5-acb4-d4a9257b3312 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:36:11.859557 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:11.859557 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:11.859557 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:11.859557 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:11.859557 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:11.859557 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:11.859557 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:11.859557 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:11.859557 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:11.859557 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:11.859557 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:11.859557 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:36:11.859557 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:36:11.859557 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:36:11.859557 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:36:11.860583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:11.860583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:11.860583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:36:11.860583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:11.860583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:36:11.860583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:11.860583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:11.860583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:11.860583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:11.860583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:11.860583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:11.860583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:11.860583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:11.860583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:11.860583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:11.860583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:11.860583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:11.860583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:11.861643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:11.861643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:11.861643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:36:11.861643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:36:11.861643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:36:11.861643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:36:11.861643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:36:11.861643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:36:11.861643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:36:11.861643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:11.861643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 361/736] 10.222.0.115 () {56 vars in 1098 bytes} [Tue Aug 2 10:36:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:11.867360 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:11.867645 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da607312-8bee-47c5-acb4-d4a9257b3312 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:11.899388 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:11.918811 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3eeef0-d9f7-4daf-bd92-37ece2c56d4d None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:11.963800 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86020436-c22a-45a7-8d44-813c6149c539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:11.964713 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86020436-c22a-45a7-8d44-813c6149c539 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:11.964995 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86020436-c22a-45a7-8d44-813c6149c539 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:11.965265 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86020436-c22a-45a7-8d44-813c6149c539 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:11.981636 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86020436-c22a-45a7-8d44-813c6149c539 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:11.983183 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86020436-c22a-45a7-8d44-813c6149c539 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:11.986960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da607312-8bee-47c5-acb4-d4a9257b3312 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:12.001878 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-86020436-c22a-45a7-8d44-813c6149c539 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:12.001878 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:12.001878 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:12.001878 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:12.001878 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:12.001878 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:12.001878 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:12.001878 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:12.001878 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:12.001878 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:12.001878 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:12.001878 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:12.001878 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:12.001878 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:12.001878 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:12.003243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:12.003243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:12.003243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:12.003243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:12.003243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:12.003243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:12.003243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:12.003243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:12.003243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:12.003243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:12.003243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:12.003243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:12.003243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:12.003243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:12.003243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:12.003243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:12.003243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:12.004393 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:12.004393 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:12.004393 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:12.004393 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:12.004393 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 376/737] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:12.008188 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db248a-ed9d-4841-884c-45ed67b0e78a None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:12.009467 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db248a-ed9d-4841-884c-45ed67b0e78a None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:12.010017 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0db248a-ed9d-4841-884c-45ed67b0e78a None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:12.011558 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 377/738] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:12 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:12.012639 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db248a-ed9d-4841-884c-45ed67b0e78a None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:12.047859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f706b43-c658-44af-a3c1-951500f50a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:12.049282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f706b43-c658-44af-a3c1-951500f50a31 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:12.049790 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f706b43-c658-44af-a3c1-951500f50a31 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:12.050314 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f706b43-c658-44af-a3c1-951500f50a31 None admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:12.051392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f706b43-c658-44af-a3c1-951500f50a31 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:12.051944 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f706b43-c658-44af-a3c1-951500f50a31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:12.054483 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f706b43-c658-44af-a3c1-951500f50a31 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:12.075135 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 362/739] 10.222.0.115 () {56 vars in 1138 bytes} [Tue Aug 2 10:36:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:36:12.077727 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee4602e-0546-41ef-8829-db935ed1acd0 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:12.078314 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee4602e-0546-41ef-8829-db935ed1acd0 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:12.078811 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee4602e-0546-41ef-8829-db935ed1acd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:12.098780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-fee4602e-0546-41ef-8829-db935ed1acd0 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:12.100402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-fee4602e-0546-41ef-8829-db935ed1acd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:12.131615 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0db248a-ed9d-4841-884c-45ed67b0e78a None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:12.143894 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:12.168852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23adfa39-25df-4f29-a169-175ec368e447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:12.169477 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23adfa39-25df-4f29-a169-175ec368e447 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:12.169899 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23adfa39-25df-4f29-a169-175ec368e447 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:12.170113 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23adfa39-25df-4f29-a169-175ec368e447 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:12.170906 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23adfa39-25df-4f29-a169-175ec368e447 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:12.171551 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23adfa39-25df-4f29-a169-175ec368e447 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:12.172727 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23adfa39-25df-4f29-a169-175ec368e447 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:12.177091 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 363/740] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:12 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:12.178239 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee4602e-0546-41ef-8829-db935ed1acd0 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:12.192558 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 378/741] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:12 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:12.196040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23adfa39-25df-4f29-a169-175ec368e447 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:12.221189 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:12.343780 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23adfa39-25df-4f29-a169-175ec368e447 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:12.346157 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee4602e-0546-41ef-8829-db935ed1acd0 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:12.384084 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cbe31d-d9de-4364-a350-b9a4ad371ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:12.385166 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cbe31d-d9de-4364-a350-b9a4ad371ddc None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:12.385669 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cbe31d-d9de-4364-a350-b9a4ad371ddc None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:12.386151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cbe31d-d9de-4364-a350-b9a4ad371ddc None admin] PATH_INFO: `/v3/users/swift` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:12.387028 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912b6f1-5fda-4347-ab0f-22c80950343a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:12.387809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912b6f1-5fda-4347-ab0f-22c80950343a None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:12.388105 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912b6f1-5fda-4347-ab0f-22c80950343a None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:12.388375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3912b6f1-5fda-4347-ab0f-22c80950343a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:12.397298 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cbe31d-d9de-4364-a350-b9a4ad371ddc None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:12.399137 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cbe31d-d9de-4364-a350-b9a4ad371ddc None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:12.400860 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3912b6f1-5fda-4347-ab0f-22c80950343a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:12.402827 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3912b6f1-5fda-4347-ab0f-22c80950343a None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:12.413161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-89cbe31d-d9de-4364-a350-b9a4ad371ddc None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:36:12.413161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:12.413161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:12.413161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:12.413161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:12.413161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:12.413161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:12.413161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:12.413161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:12.413161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:12.413161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:12.413161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:12.413161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:36:12.413161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:36:12.413161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:36:12.413161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:36:12.414190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:12.414190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:12.414190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:36:12.414190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:12.414190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:36:12.414190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:12.414190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:12.414190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:12.414190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:12.414190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:12.414190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:12.414190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:12.414190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:12.414190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:12.414190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:12.414190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:12.414190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:12.414190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:12.415178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:12.415178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:12.415178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:36:12.415178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:36:12.415178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:36:12.415178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:36:12.415178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:36:12.415178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:36:12.415178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:36:12.415178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:12.422262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-3912b6f1-5fda-4347-ab0f-22c80950343a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:12.422262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:12.422262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:12.422262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:12.422262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:12.422262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:12.422262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:12.422262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:12.422262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:12.422262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:12.422262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:12.422262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:12.422262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:12.422262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:12.422262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:12.423262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:12.423262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:12.423262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:12.423262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:12.423262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:12.423262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:12.423262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:12.423262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:12.423262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:12.423262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:12.423262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:12.423262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:12.423262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:12.423262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:12.423262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:12.423262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:12.423262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:12.424166 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:12.424166 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:12.424166 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:12.424166 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:12.424166 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 364/743] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:12.424166 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912b6f1-5fda-4347-ab0f-22c80950343a None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:12.425002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 379/743] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:36:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:12.426142 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cbe31d-d9de-4364-a350-b9a4ad371ddc None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:12.489870 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:12.555590 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:12.606556 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cbe31d-d9de-4364-a350-b9a4ad371ddc None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:12.607027 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3912b6f1-5fda-4347-ab0f-22c80950343a None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:12.651040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42d74d-ffe7-45db-a870-80cc730e078c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:12.652144 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42d74d-ffe7-45db-a870-80cc730e078c None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:12.652627 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42d74d-ffe7-45db-a870-80cc730e078c None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:12.653099 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42d74d-ffe7-45db-a870-80cc730e078c None admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:12.654104 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a42d74d-ffe7-45db-a870-80cc730e078c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:12.654698 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a42d74d-ffe7-45db-a870-80cc730e078c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:12.656334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7d44c-2261-4fa6-95cc-8cdb7a5067cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:12.657066 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a42d74d-ffe7-45db-a870-80cc730e078c None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:12.657419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7d44c-2261-4fa6-95cc-8cdb7a5067cd None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:12.657911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7d44c-2261-4fa6-95cc-8cdb7a5067cd None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:12.658431 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7d44c-2261-4fa6-95cc-8cdb7a5067cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:12.679506 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f7d44c-2261-4fa6-95cc-8cdb7a5067cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:36:12.685887 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 380/744] 10.222.0.115 () {56 vars in 1140 bytes} [Tue Aug 2 10:36:12 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:12.687547 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42d74d-ffe7-45db-a870-80cc730e078c None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:12.690633 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-e2f7d44c-2261-4fa6-95cc-8cdb7a5067cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:36:12.690633 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:12.690633 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:12.690633 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:12.690633 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:12.690633 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:12.690633 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:12.690633 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:12.690633 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:12.690633 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:12.690633 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:12.690633 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:12.690633 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:36:12.690633 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:36:12.690633 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:36:12.690633 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:36:12.691630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:36:12.691630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:36:12.691630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:36:12.691630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:36:12.691630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:12.691630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:12.691630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:12.691630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:12.691630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:12.691630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:12.691630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:12.691630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:12.691630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:12.691630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:12.691630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:12.691630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:12.692466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:12.692466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:12.692466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:12.692466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:12.692466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:36:12.692466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:36:12.692466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:36:12.692466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:36:12.692466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:36:12.692466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:36:12.692466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:36:12.692466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:12.693112 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 365/745] 10.222.0.115 () {56 vars in 1107 bytes} [Tue Aug 2 10:36:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:12.694429 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7d44c-2261-4fa6-95cc-8cdb7a5067cd None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:12.805575 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:12.821807 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42d74d-ffe7-45db-a870-80cc730e078c None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:12.836194 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7d44c-2261-4fa6-95cc-8cdb7a5067cd None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:12.866823 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff4220-a5d3-4521-aa22-764b678c82b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:12.867492 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff4220-a5d3-4521-aa22-764b678c82b9 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:12.867772 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff4220-a5d3-4521-aa22-764b678c82b9 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:12.868045 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abff4220-a5d3-4521-aa22-764b678c82b9 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:12.868748 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff4220-a5d3-4521-aa22-764b678c82b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:12.869050 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff4220-a5d3-4521-aa22-764b678c82b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:12.870496 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abff4220-a5d3-4521-aa22-764b678c82b9 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:12.876992 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:12.885423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 381/746] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:12 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:12.886174 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff4220-a5d3-4521-aa22-764b678c82b9 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:12.905856 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029971d-332d-4493-a54c-0403c26609cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:12.905856 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029971d-332d-4493-a54c-0403c26609cf None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:12.905856 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029971d-332d-4493-a54c-0403c26609cf None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:12.905856 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2029971d-332d-4493-a54c-0403c26609cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:12.980805 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2029971d-332d-4493-a54c-0403c26609cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:12.982287 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2029971d-332d-4493-a54c-0403c26609cf None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:13.009650 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abff4220-a5d3-4521-aa22-764b678c82b9 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:13.059571 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930e2aa2-0ab0-4286-a766-13959d72c22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:13.060561 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e2aa2-0ab0-4286-a766-13959d72c22f None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:13.061275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e2aa2-0ab0-4286-a766-13959d72c22f None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:13.061563 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930e2aa2-0ab0-4286-a766-13959d72c22f None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:13.062291 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930e2aa2-0ab0-4286-a766-13959d72c22f None admin] RBAC: Adding query filter params (name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:13.062600 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930e2aa2-0ab0-4286-a766-13959d72c22f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:13.064133 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930e2aa2-0ab0-4286-a766-13959d72c22f None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:13.075373 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 382/747] 10.222.0.115 () {56 vars in 1108 bytes} [Tue Aug 2 10:36:12 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:13.077020 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2b5a2-0cee-458b-9388-3b1323e0d136 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:13.077487 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2b5a2-0cee-458b-9388-3b1323e0d136 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:13.077768 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2b5a2-0cee-458b-9388-3b1323e0d136 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:13.079307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 383/748] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:13.080178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b2b5a2-0cee-458b-9388-3b1323e0d136 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:13.094184 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:13.105833 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 366/749] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:36:12 2022] GET /identity/v3/auth/tokens => generated 1003 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:36:13.106769 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029971d-332d-4493-a54c-0403c26609cf None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:13.142008 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:13.202559 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b2b5a2-0cee-458b-9388-3b1323e0d136 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:13.228968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2029971d-332d-4493-a54c-0403c26609cf None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:13.249070 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09383923-4cea-47eb-8486-8706bf46975d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:13.250214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09383923-4cea-47eb-8486-8706bf46975d None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:13.250490 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09383923-4cea-47eb-8486-8706bf46975d None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:13.250763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09383923-4cea-47eb-8486-8706bf46975d None admin] PATH_INFO: `/v3/projects/service` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:13.258664 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09383923-4cea-47eb-8486-8706bf46975d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:13.260179 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09383923-4cea-47eb-8486-8706bf46975d None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:13.273903 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-09383923-4cea-47eb-8486-8706bf46975d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:13.273903 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:13.273903 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:13.273903 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:13.273903 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:13.273903 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:13.273903 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:13.273903 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:13.273903 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:13.273903 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:13.273903 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:13.273903 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:13.273903 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:36:13.273903 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:36:13.273903 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:36:13.275787 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:36:13.275787 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:13.275787 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:13.275787 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:13.275787 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:13.275787 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:13.275787 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:13.275787 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:13.275787 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:13.275787 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:13.275787 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:13.275787 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:13.275787 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:13.275787 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:13.275787 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:13.275787 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:13.275787 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:13.276704 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:36:13.276704 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:36:13.276704 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:36:13.276704 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:36:13.276704 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:36:13.276704 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:36:13.276704 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:13.276704 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:13.276704 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 384/750] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:36:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:13.276704 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d4013-6db9-4533-975b-989062457c13 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:13.276704 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d4013-6db9-4533-975b-989062457c13 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:13.277327 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644d4013-6db9-4533-975b-989062457c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:13.291921 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a926d9f8-d90b-4769-98ed-c4139ca8012a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:13.292991 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926d9f8-d90b-4769-98ed-c4139ca8012a None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:13.293520 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926d9f8-d90b-4769-98ed-c4139ca8012a None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:13.294038 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a926d9f8-d90b-4769-98ed-c4139ca8012a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:13.301089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-644d4013-6db9-4533-975b-989062457c13 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:13.302879 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-644d4013-6db9-4533-975b-989062457c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:13.305625 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a926d9f8-d90b-4769-98ed-c4139ca8012a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:13.307183 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a926d9f8-d90b-4769-98ed-c4139ca8012a None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:13.323715 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-a926d9f8-d90b-4769-98ed-c4139ca8012a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:13.323715 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:13.323715 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:13.323715 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:13.323715 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:13.323715 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:13.323715 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:13.323715 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:13.323715 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:13.323715 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:13.323715 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:13.323715 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:13.323715 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:13.323715 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:13.323715 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:13.324699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:13.324699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:13.324699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:13.324699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:13.324699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:13.324699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:13.324699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:13.324699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:13.324699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:13.324699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:13.324699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:13.324699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:13.324699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:13.324699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:13.324699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:13.324699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:13.324699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:13.325630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:13.325630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:13.325630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:13.325630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:13.330080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 367/751] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:13.331274 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a926d9f8-d90b-4769-98ed-c4139ca8012a None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:13.396028 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:13.400876 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 385/752] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:13 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:13.401514 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d4013-6db9-4533-975b-989062457c13 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:13.457035 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:13.490271 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a926d9f8-d90b-4769-98ed-c4139ca8012a None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:13.532583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb470585-990b-40ba-9d31-c094a8a53ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:13.533201 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb470585-990b-40ba-9d31-c094a8a53ddd None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:13.533477 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb470585-990b-40ba-9d31-c094a8a53ddd None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:13.533748 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb470585-990b-40ba-9d31-c094a8a53ddd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:13.535544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644d4013-6db9-4533-975b-989062457c13 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:13.545827 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb470585-990b-40ba-9d31-c094a8a53ddd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:13.547509 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb470585-990b-40ba-9d31-c094a8a53ddd None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:13.568850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-cb470585-990b-40ba-9d31-c094a8a53ddd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:13.568850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:13.568850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:13.568850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:13.568850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:13.568850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:13.568850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:13.568850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:13.568850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:13.568850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:13.568850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:13.568850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:13.568850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:13.568850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:13.568850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:13.570193 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:13.570193 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:13.570193 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:13.570193 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:13.570193 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:13.570193 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:13.570193 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:13.570193 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:13.570193 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:13.570193 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:13.570193 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:13.570193 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:13.570193 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:13.570193 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:13.570193 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:13.570193 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:13.570193 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:13.571129 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:13.571129 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:13.571129 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:13.571129 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:13.571129 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 368/753] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:13.572197 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb470585-990b-40ba-9d31-c094a8a53ddd None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:13.586987 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515635f1-f28a-4b13-8805-e663c9911e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:13.587838 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515635f1-f28a-4b13-8805-e663c9911e8c None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:13.588001 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515635f1-f28a-4b13-8805-e663c9911e8c None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:13.588241 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515635f1-f28a-4b13-8805-e663c9911e8c None admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:13.588987 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515635f1-f28a-4b13-8805-e663c9911e8c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:13.589292 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515635f1-f28a-4b13-8805-e663c9911e8c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:13.590936 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515635f1-f28a-4b13-8805-e663c9911e8c None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:13.610429 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 386/754] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:36:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:13.610429 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515635f1-f28a-4b13-8805-e663c9911e8c None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:13.683648 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:13.726475 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb470585-990b-40ba-9d31-c094a8a53ddd None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:13.743265 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515635f1-f28a-4b13-8805-e663c9911e8c None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:13.776715 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:13.783071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2a8bc7-15a8-47d6-ad7b-9111dfc43cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:13.783729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a8bc7-15a8-47d6-ad7b-9111dfc43cc7 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:13.784013 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a8bc7-15a8-47d6-ad7b-9111dfc43cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:13.784112 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6cbd25-a00e-4575-a0a5-8b0f435b17f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:13.786879 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2a8bc7-15a8-47d6-ad7b-9111dfc43cc7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:13.786999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6cbd25-a00e-4575-a0a5-8b0f435b17f7 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:13.787107 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2a8bc7-15a8-47d6-ad7b-9111dfc43cc7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:13.787190 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6cbd25-a00e-4575-a0a5-8b0f435b17f7 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:13.787190 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6cbd25-a00e-4575-a0a5-8b0f435b17f7 None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:13.787190 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6cbd25-a00e-4575-a0a5-8b0f435b17f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:13.787190 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6cbd25-a00e-4575-a0a5-8b0f435b17f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:13.787463 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2a8bc7-15a8-47d6-ad7b-9111dfc43cc7 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:13.787808 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6cbd25-a00e-4575-a0a5-8b0f435b17f7 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:13.797588 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-1a2a8bc7-15a8-47d6-ad7b-9111dfc43cc7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 10:36:13.797588 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:13.797588 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:36:13.797588 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:36:13.797588 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:36:13.797588 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:36:13.797588 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:36:13.797588 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:13.797588 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 10:36:13.797588 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:13.797588 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:36:13.797588 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:13.797588 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:13.797588 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:13.797588 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:13.797588 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:13.799426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:13.799426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:13.799426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:13.799426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:13.799426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:13.799426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:13.799426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:13.799426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:36:13.799426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:36:13.799426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:36:13.799426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:36:13.799426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:13.799426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:13.799426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:13.799426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:13.799426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:13.801183 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:13.801183 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:13.801183 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:13.801183 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:13.801183 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:13.801183 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:13.801183 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:13.801183 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:13.801183 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:13.801183 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:13.801183 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:13.801183 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:36:13.801183 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:13.801183 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 10:36:13.801183 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:13.801183 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 387/755] 10.222.0.115 () {56 vars in 1134 bytes} [Tue Aug 2 10:36:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:13.802848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2a8bc7-15a8-47d6-ad7b-9111dfc43cc7 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:13.809046 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 369/756] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:13.809046 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6cbd25-a00e-4575-a0a5-8b0f435b17f7 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:13.938811 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6cbd25-a00e-4575-a0a5-8b0f435b17f7 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:13.944458 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2a8bc7-15a8-47d6-ad7b-9111dfc43cc7 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:13.972041 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:13.984873 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e7b39-1c84-4a4e-9844-6d892cf958cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:13.984873 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e7b39-1c84-4a4e-9844-6d892cf958cc None admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:13.984873 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e7b39-1c84-4a4e-9844-6d892cf958cc None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:13.984873 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75e7b39-1c84-4a4e-9844-6d892cf958cc None admin] PATH_INFO: `/v3/projects/c1eae881e5714bf4bd223809e1a2d920/users/d0478bd540f446ec810e2091c46ac49c/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:13.987580 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75e7b39-1c84-4a4e-9844-6d892cf958cc None admin] RBAC: Authorizing `identity:create_grant(project_id=c1eae881e5714bf4bd223809e1a2d920, user_id=d0478bd540f446ec810e2091c46ac49c, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:13.989141 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75e7b39-1c84-4a4e-9844-6d892cf958cc None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:13.990068 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8938a6-e83a-49aa-a5f2-cf50eb2e43c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:13.990699 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8938a6-e83a-49aa-a5f2-cf50eb2e43c7 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:13.990978 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8938a6-e83a-49aa-a5f2-cf50eb2e43c7 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:13.991278 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8938a6-e83a-49aa-a5f2-cf50eb2e43c7 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:13.991982 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8938a6-e83a-49aa-a5f2-cf50eb2e43c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:13.992286 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8938a6-e83a-49aa-a5f2-cf50eb2e43c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:13.993645 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8938a6-e83a-49aa-a5f2-cf50eb2e43c7 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:14.007273 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 388/757] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:13 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:14.008081 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8938a6-e83a-49aa-a5f2-cf50eb2e43c7 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:14.101584 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 370/758] 10.222.0.115 () {58 vars in 1444 bytes} [Tue Aug 2 10:36:13 2022] PUT /identity/v3/projects/c1eae881e5714bf4bd223809e1a2d920/users/d0478bd540f446ec810e2091c46ac49c/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:36:14.103123 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e7b39-1c84-4a4e-9844-6d892cf958cc None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:14.137544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8938a6-e83a-49aa-a5f2-cf50eb2e43c7 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:14.181395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:14.189555 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98196a3e-8f2f-48d3-a6b0-9dfd2b6f58a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:14.190196 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98196a3e-8f2f-48d3-a6b0-9dfd2b6f58a1 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:14.190568 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98196a3e-8f2f-48d3-a6b0-9dfd2b6f58a1 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:14.190840 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98196a3e-8f2f-48d3-a6b0-9dfd2b6f58a1 None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:14.191581 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98196a3e-8f2f-48d3-a6b0-9dfd2b6f58a1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:14.191879 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98196a3e-8f2f-48d3-a6b0-9dfd2b6f58a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:14.193309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98196a3e-8f2f-48d3-a6b0-9dfd2b6f58a1 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:14.204416 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 389/759] 10.222.0.115 () {56 vars in 1129 bytes} [Tue Aug 2 10:36:14 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:14.204416 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fc676-9b7b-4d2a-b17b-5228e7ba40d6 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:14.204416 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fc676-9b7b-4d2a-b17b-5228e7ba40d6 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:14.204416 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fc676-9b7b-4d2a-b17b-5228e7ba40d6 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:14.207210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 390/760] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:14 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:14.208451 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26fc676-9b7b-4d2a-b17b-5228e7ba40d6 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:14.250523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75e7b39-1c84-4a4e-9844-6d892cf958cc None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:14.272361 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:14.293053 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339d01e-e0dc-49fd-b32d-21adaf631941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:14.293893 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339d01e-e0dc-49fd-b32d-21adaf631941 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:14.294171 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339d01e-e0dc-49fd-b32d-21adaf631941 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:14.294435 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339d01e-e0dc-49fd-b32d-21adaf631941 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:14.306188 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5339d01e-e0dc-49fd-b32d-21adaf631941 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:14.308249 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5339d01e-e0dc-49fd-b32d-21adaf631941 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:14.321440 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-5339d01e-e0dc-49fd-b32d-21adaf631941 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:36:14.321440 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:14.321440 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:14.321440 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:14.321440 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:14.321440 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:14.321440 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:14.321440 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:14.321440 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:14.321440 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:14.321440 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:14.321440 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:14.321440 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:36:14.321440 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:36:14.321440 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:36:14.321440 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:36:14.322469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:14.322469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:14.322469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:36:14.322469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:14.322469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:36:14.322469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:14.322469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:14.322469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:14.322469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:14.322469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:14.322469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:14.322469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:14.322469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:14.322469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:14.322469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:14.322469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:14.322469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:14.322469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:14.323475 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:14.323475 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:14.323475 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:36:14.323475 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:36:14.323475 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:36:14.323475 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:36:14.323475 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:36:14.323475 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:36:14.323475 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:36:14.323475 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:14.323475 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 371/761] 10.222.0.115 () {56 vars in 1122 bytes} [Tue Aug 2 10:36:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:14.323475 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339d01e-e0dc-49fd-b32d-21adaf631941 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:14.345444 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26fc676-9b7b-4d2a-b17b-5228e7ba40d6 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:14.396545 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c754795a-a929-4fc0-9aa9-87475bca7941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:14.397013 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754795a-a929-4fc0-9aa9-87475bca7941 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:14.397322 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754795a-a929-4fc0-9aa9-87475bca7941 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:14.397709 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c754795a-a929-4fc0-9aa9-87475bca7941 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:14.407527 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c754795a-a929-4fc0-9aa9-87475bca7941 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:14.408987 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c754795a-a929-4fc0-9aa9-87475bca7941 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:14.424014 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-c754795a-a929-4fc0-9aa9-87475bca7941 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:14.424014 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:14.424014 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:14.424014 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:14.424014 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:14.424014 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:14.424014 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:14.424014 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:14.424014 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:14.424014 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:14.424014 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:14.424014 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:14.424014 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:36:14.424014 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:36:14.424014 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:36:14.425635 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:36:14.425635 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:14.425635 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:14.425635 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:14.425635 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:14.425635 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:14.425635 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:14.425635 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:14.425635 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:14.425635 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:14.425635 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:14.425635 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:14.425635 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:14.425635 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:14.425635 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:14.425635 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:14.425635 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:14.427454 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:36:14.427454 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:36:14.427454 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:36:14.427454 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:36:14.427454 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:36:14.427454 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:36:14.427454 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:14.427454 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:14.427454 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 391/762] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:36:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:14.427454 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d97927-840d-4da6-978f-2f3c1e6baf86 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:14.427454 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d97927-840d-4da6-978f-2f3c1e6baf86 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:14.428612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d97927-840d-4da6-978f-2f3c1e6baf86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:14.462959 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-32d97927-840d-4da6-978f-2f3c1e6baf86 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:14.464825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-32d97927-840d-4da6-978f-2f3c1e6baf86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:14.484183 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339d01e-e0dc-49fd-b32d-21adaf631941 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:14.498896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:14.531720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e3d94c-d8de-4ec9-85b5-00a094a0eb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:14.532359 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3d94c-d8de-4ec9-85b5-00a094a0eb09 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:14.532643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3d94c-d8de-4ec9-85b5-00a094a0eb09 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:14.532921 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3d94c-d8de-4ec9-85b5-00a094a0eb09 None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:14.533678 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e3d94c-d8de-4ec9-85b5-00a094a0eb09 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:14.533989 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e3d94c-d8de-4ec9-85b5-00a094a0eb09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:14.535413 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e3d94c-d8de-4ec9-85b5-00a094a0eb09 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:14.548743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 372/763] 10.222.0.115 () {56 vars in 1129 bytes} [Tue Aug 2 10:36:14 2022] GET /identity/v3/services?type=block-storage => generated 369 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:14.548743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e3d94c-d8de-4ec9-85b5-00a094a0eb09 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:14.561904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 392/764] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:14 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:14.568195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d97927-840d-4da6-978f-2f3c1e6baf86 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:14.622462 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:14.698706 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e3d94c-d8de-4ec9-85b5-00a094a0eb09 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:14.699133 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d97927-840d-4da6-978f-2f3c1e6baf86 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:14.737647 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250f8af-9d58-495d-a021-f9dffdcbf227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:14.738107 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250f8af-9d58-495d-a021-f9dffdcbf227 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:14.738391 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250f8af-9d58-495d-a021-f9dffdcbf227 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:14.738701 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250f8af-9d58-495d-a021-f9dffdcbf227 None admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:14.739502 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250f8af-9d58-495d-a021-f9dffdcbf227 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:14.739855 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250f8af-9d58-495d-a021-f9dffdcbf227 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:14.741423 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250f8af-9d58-495d-a021-f9dffdcbf227 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:14.742959 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa51e9-ad2b-4367-9da2-da3a56a792e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:14.743633 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa51e9-ad2b-4367-9da2-da3a56a792e5 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:14.743911 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa51e9-ad2b-4367-9da2-da3a56a792e5 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:14.744182 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa51e9-ad2b-4367-9da2-da3a56a792e5 None admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:14.744947 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa51e9-ad2b-4367-9da2-da3a56a792e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:14.745251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa51e9-ad2b-4367-9da2-da3a56a792e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:14.746705 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa51e9-ad2b-4367-9da2-da3a56a792e5 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:14.761927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 393/765] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:36:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:14.762856 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 373/766] 10.222.0.115 () {56 vars in 1154 bytes} [Tue Aug 2 10:36:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:14.763253 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa51e9-ad2b-4367-9da2-da3a56a792e5 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:14.764697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250f8af-9d58-495d-a021-f9dffdcbf227 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:14.806872 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:14.892605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250f8af-9d58-495d-a021-f9dffdcbf227 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:14.908132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:14.912604 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa51e9-ad2b-4367-9da2-da3a56a792e5 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:14.934125 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ff3c0-fb53-4389-be26-57fc8743e745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:14.934773 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ff3c0-fb53-4389-be26-57fc8743e745 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:14.935218 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ff3c0-fb53-4389-be26-57fc8743e745 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:14.935373 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08ff3c0-fb53-4389-be26-57fc8743e745 None admin] PATH_INFO: `/v3/services/network` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:14.936523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ff3c0-fb53-4389-be26-57fc8743e745 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:14.938057 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08ff3c0-fb53-4389-be26-57fc8743e745 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:14.949973 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-e08ff3c0-fb53-4389-be26-57fc8743e745 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 10:36:14.949973 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:14.949973 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:36:14.949973 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:36:14.949973 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:36:14.949973 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:36:14.949973 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:36:14.949973 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:14.949973 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 10:36:14.949973 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:14.949973 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:36:14.949973 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:14.949973 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:14.949973 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:14.949973 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:14.949973 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:14.949973 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:14.951036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:14.951036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:14.951036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:14.951036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:14.951036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:14.951036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:14.951036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:36:14.951036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:36:14.951036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:36:14.951036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:36:14.951036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:14.951036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:14.951036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:14.951036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:14.951036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:14.951036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:14.952010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:14.952010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:14.952010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:14.952010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:14.952010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:14.952010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:14.952010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:14.952010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:14.952010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:14.952010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:14.952010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:36:14.952010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:14.952010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 10:36:14.952010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:14.952010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 374/767] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:36:14 2022] GET /identity/v3/services/network => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:14.953426 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ff3c0-fb53-4389-be26-57fc8743e745 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:14.962224 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0f3e9-be11-4601-a036-bdb8a05fba95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:14.962856 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0f3e9-be11-4601-a036-bdb8a05fba95 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:14.963165 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0f3e9-be11-4601-a036-bdb8a05fba95 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:14.963445 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da0f3e9-be11-4601-a036-bdb8a05fba95 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:14.964213 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da0f3e9-be11-4601-a036-bdb8a05fba95 None admin] RBAC: Adding query filter params (interface=public, service_id=b252d5f130004c83bbf168403478e21e, region_id=RegionOne) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:14.964529 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da0f3e9-be11-4601-a036-bdb8a05fba95 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:14.965909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da0f3e9-be11-4601-a036-bdb8a05fba95 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:14.973813 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 394/768] 10.222.0.115 () {56 vars in 1256 bytes} [Tue Aug 2 10:36:14 2022] GET /identity/v3/endpoints?service_id=b252d5f130004c83bbf168403478e21e&interface=public®ion_id=RegionOne => generated 195 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:14.979038 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0f3e9-be11-4601-a036-bdb8a05fba95 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:15.064081 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:15.072080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08ff3c0-fb53-4389-be26-57fc8743e745 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:15.090700 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da0f3e9-be11-4601-a036-bdb8a05fba95 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:15.120844 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1fe46-a809-4809-a2a2-d0d6cdac627a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:15.121472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1fe46-a809-4809-a2a2-d0d6cdac627a None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:15.121738 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1fe46-a809-4809-a2a2-d0d6cdac627a None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:15.122011 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1fe46-a809-4809-a2a2-d0d6cdac627a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:15.139807 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46edc0b3-8a32-402e-a697-3d85ef4b59f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:15.141127 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edc0b3-8a32-402e-a697-3d85ef4b59f1 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:15.141599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edc0b3-8a32-402e-a697-3d85ef4b59f1 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:15.142063 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46edc0b3-8a32-402e-a697-3d85ef4b59f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:15.143035 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46edc0b3-8a32-402e-a697-3d85ef4b59f1 None admin] RBAC: Adding query filter params (role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac, scope.project.id=c1eae881e5714bf4bd223809e1a2d920, user.id=0daa49b5a35f4a468bb8d0939e7ec20c) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:15.143597 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46edc0b3-8a32-402e-a697-3d85ef4b59f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:15.145365 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46edc0b3-8a32-402e-a697-3d85ef4b59f1 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:15.151530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 395/769] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:36:14 2022] GET /identity/v3/role_assignments?user.id=0daa49b5a35f4a468bb8d0939e7ec20c&scope.project.id=c1eae881e5714bf4bd223809e1a2d920&role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:15.152157 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46edc0b3-8a32-402e-a697-3d85ef4b59f1 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:15.192973 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:15.232810 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef1fe46-a809-4809-a2a2-d0d6cdac627a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:15.236097 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef1fe46-a809-4809-a2a2-d0d6cdac627a None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:15.323717 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46edc0b3-8a32-402e-a697-3d85ef4b59f1 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:15.368976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e32773f-c689-4fe8-a650-733d53412e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:15.370155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32773f-c689-4fe8-a650-733d53412e87 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:15.370643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32773f-c689-4fe8-a650-733d53412e87 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:15.371249 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e32773f-c689-4fe8-a650-733d53412e87 None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:15.372425 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e32773f-c689-4fe8-a650-733d53412e87 None admin] RBAC: Adding query filter params (name=network) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:15.373503 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e32773f-c689-4fe8-a650-733d53412e87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:15.375604 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e32773f-c689-4fe8-a650-733d53412e87 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:15.386742 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 396/770] 10.222.0.115 () {56 vars in 1117 bytes} [Tue Aug 2 10:36:15 2022] GET /identity/v3/services?name=network => generated 125 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:15.387981 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e32773f-c689-4fe8-a650-733d53412e87 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:15.396904 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:15.401760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 375/771] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:36:14 2022] GET /identity/v3/auth/tokens => generated 1003 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:36:15.402790 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1fe46-a809-4809-a2a2-d0d6cdac627a None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:15.508857 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:15.521449 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e32773f-c689-4fe8-a650-733d53412e87 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:15.566548 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d8d01-6ff1-46dc-b942-df83e642ca6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:15.567205 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d8d01-6ff1-46dc-b942-df83e642ca6b None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:15.567486 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d8d01-6ff1-46dc-b942-df83e642ca6b None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:15.567763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705d8d01-6ff1-46dc-b942-df83e642ca6b None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:15.568417 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d8d01-6ff1-46dc-b942-df83e642ca6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:15.570111 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705d8d01-6ff1-46dc-b942-df83e642ca6b None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:15.579296 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 397/772] 10.222.0.115 () {56 vars in 1092 bytes} [Tue Aug 2 10:36:15 2022] GET /identity/v3/services => generated 776 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:15.580935 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d8d01-6ff1-46dc-b942-df83e642ca6b None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:15.602390 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1fe46-a809-4809-a2a2-d0d6cdac627a None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:15.681253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeeab54-68a6-4d29-b743-a6ae462606b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:15.682280 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeeab54-68a6-4d29-b743-a6ae462606b5 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:15.683039 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeeab54-68a6-4d29-b743-a6ae462606b5 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:15.683657 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeeab54-68a6-4d29-b743-a6ae462606b5 None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:15.686327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeeab54-68a6-4d29-b743-a6ae462606b5 None admin] RBAC: Adding query filter params (type=network) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:15.686991 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeeab54-68a6-4d29-b743-a6ae462606b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:15.689007 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeeab54-68a6-4d29-b743-a6ae462606b5 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:15.705202 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 376/773] 10.222.0.115 () {56 vars in 1117 bytes} [Tue Aug 2 10:36:15 2022] GET /identity/v3/services?type=network => generated 352 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:15.712360 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeeab54-68a6-4d29-b743-a6ae462606b5 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:15.801758 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:15.831800 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705d8d01-6ff1-46dc-b942-df83e642ca6b None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:15.865216 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeeab54-68a6-4d29-b743-a6ae462606b5 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:15.883962 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67168049-0610-4fbb-8cc0-c811f17446e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:15.884990 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67168049-0610-4fbb-8cc0-c811f17446e7 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:15.885509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67168049-0610-4fbb-8cc0-c811f17446e7 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:15.885985 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67168049-0610-4fbb-8cc0-c811f17446e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:15.902249 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67168049-0610-4fbb-8cc0-c811f17446e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:15.905281 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67168049-0610-4fbb-8cc0-c811f17446e7 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:15.925609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79052b-7f55-4d80-a6ff-6761e8335f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:15.926091 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-67168049-0610-4fbb-8cc0-c811f17446e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:15.926091 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:15.926091 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:15.926091 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:15.926091 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:15.926091 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:15.926091 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:15.926091 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:15.926091 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:15.926091 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:15.926091 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:15.926091 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:15.926091 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:15.926091 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:15.926091 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:15.926941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:15.926941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:15.926941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:15.926941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:15.926941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:15.926941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:15.926941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:15.926941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:15.926941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:15.926941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:15.926941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:15.926941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:15.926941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:15.926941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:15.926941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:15.926941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:15.926941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:15.928131 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:15.928131 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:15.928131 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:15.928131 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:15.928352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79052b-7f55-4d80-a6ff-6761e8335f84 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:15.928352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79052b-7f55-4d80-a6ff-6761e8335f84 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:15.928352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f79052b-7f55-4d80-a6ff-6761e8335f84 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:15.928352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79052b-7f55-4d80-a6ff-6761e8335f84 None admin] RBAC: Adding query filter params (interface=public, service_id=90f0a0c74d194686a1dae979ef844874, region_id=RegionOne) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:15.928595 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79052b-7f55-4d80-a6ff-6761e8335f84 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:15.930080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f79052b-7f55-4d80-a6ff-6761e8335f84 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:15.930707 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 398/774] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:15.936917 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67168049-0610-4fbb-8cc0-c811f17446e7 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:15.942567 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 377/775] 10.222.0.115 () {56 vars in 1256 bytes} [Tue Aug 2 10:36:15 2022] GET /identity/v3/endpoints?service_id=90f0a0c74d194686a1dae979ef844874&interface=public®ion_id=RegionOne => generated 195 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:15.952889 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79052b-7f55-4d80-a6ff-6761e8335f84 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:15.964279 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:16.058452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67168049-0610-4fbb-8cc0-c811f17446e7 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:16.099547 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b83807-1a08-4da0-9e90-680022941714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:16.100099 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b83807-1a08-4da0-9e90-680022941714 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:16.100380 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b83807-1a08-4da0-9e90-680022941714 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:16.100654 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b83807-1a08-4da0-9e90-680022941714 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:16.101368 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b83807-1a08-4da0-9e90-680022941714 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:16.101672 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b83807-1a08-4da0-9e90-680022941714 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:16.103071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b83807-1a08-4da0-9e90-680022941714 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:16.117645 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 399/776] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:16.124105 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b83807-1a08-4da0-9e90-680022941714 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:16.203310 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:16.236123 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f79052b-7f55-4d80-a6ff-6761e8335f84 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:16.279362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b83807-1a08-4da0-9e90-680022941714 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:16.291578 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:16.312806 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd351d6-800e-438a-b0c6-816ce242e019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:16.313442 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd351d6-800e-438a-b0c6-816ce242e019 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:16.313723 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd351d6-800e-438a-b0c6-816ce242e019 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:16.313995 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd351d6-800e-438a-b0c6-816ce242e019 None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:16.314653 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd351d6-800e-438a-b0c6-816ce242e019 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:16.316627 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd351d6-800e-438a-b0c6-816ce242e019 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:16.322947 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 378/777] 10.222.0.115 () {56 vars in 1092 bytes} [Tue Aug 2 10:36:15 2022] GET /identity/v3/services => generated 776 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:16.347815 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf701cca-4c5f-4308-b8e9-d0dafb0bf323 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:16.348246 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf701cca-4c5f-4308-b8e9-d0dafb0bf323 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:16.348432 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf701cca-4c5f-4308-b8e9-d0dafb0bf323 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:16.351505 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 379/778] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:16 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:16.365941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599057f8-1faf-451e-a895-0a93d57dad06 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:16.366328 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599057f8-1faf-451e-a895-0a93d57dad06 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:16.366632 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599057f8-1faf-451e-a895-0a93d57dad06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:16.369182 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1695de6-2041-43a4-b9a9-06d217a70fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:16.369841 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1695de6-2041-43a4-b9a9-06d217a70fea None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:16.370114 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1695de6-2041-43a4-b9a9-06d217a70fea None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:16.370381 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1695de6-2041-43a4-b9a9-06d217a70fea None admin] PATH_INFO: `/v3/projects/service` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:16.380020 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1695de6-2041-43a4-b9a9-06d217a70fea None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:16.381490 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1695de6-2041-43a4-b9a9-06d217a70fea None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:16.408576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-b1695de6-2041-43a4-b9a9-06d217a70fea None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:16.408576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:16.408576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:16.408576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:16.408576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:16.408576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:16.408576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:16.408576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:16.408576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:16.408576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:16.408576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:16.408576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:16.408576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:36:16.408576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:36:16.408576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:36:16.409530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:36:16.409530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:16.409530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:16.409530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:16.409530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:16.409530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:16.409530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:16.409530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:16.409530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:16.409530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:16.409530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:16.409530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:16.409530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:16.409530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:16.409530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:16.409530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:16.409530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:16.410420 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:36:16.410420 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:36:16.410420 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:36:16.410420 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:36:16.410420 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:36:16.410420 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:36:16.410420 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:16.410420 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:16.411415 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-599057f8-1faf-451e-a895-0a93d57dad06 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:16.412455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-599057f8-1faf-451e-a895-0a93d57dad06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:16.414689 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 400/779] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:36:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:16.415788 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1695de6-2041-43a4-b9a9-06d217a70fea None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:16.567153 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 380/780] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:16 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:16.593896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599057f8-1faf-451e-a895-0a93d57dad06 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:16.627181 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1695de6-2041-43a4-b9a9-06d217a70fea None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:16.663510 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:16.682306 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181af48e-5c05-4f63-952f-3e542b3eebde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:16.682937 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181af48e-5c05-4f63-952f-3e542b3eebde None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:16.683247 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181af48e-5c05-4f63-952f-3e542b3eebde None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:16.683576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181af48e-5c05-4f63-952f-3e542b3eebde None admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:16.684394 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181af48e-5c05-4f63-952f-3e542b3eebde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:16.684705 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181af48e-5c05-4f63-952f-3e542b3eebde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:16.687519 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181af48e-5c05-4f63-952f-3e542b3eebde None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:16.710514 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 401/781] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:36:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:16.718709 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181af48e-5c05-4f63-952f-3e542b3eebde None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:16.787479 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599057f8-1faf-451e-a895-0a93d57dad06 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:16.828201 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:16.854258 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b7850-c9da-4298-9414-3e8d07378e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:16.855117 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b7850-c9da-4298-9414-3e8d07378e04 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:16.855422 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b7850-c9da-4298-9414-3e8d07378e04 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:16.855837 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92b7850-c9da-4298-9414-3e8d07378e04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:16.876439 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92b7850-c9da-4298-9414-3e8d07378e04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:36:16.883718 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-f92b7850-c9da-4298-9414-3e8d07378e04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:36:16.883718 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:16.883718 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:16.883718 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:16.883718 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:16.883718 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:16.883718 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:16.883718 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:16.883718 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:16.883718 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:16.883718 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:16.883718 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:16.883718 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:36:16.883718 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:36:16.883718 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:36:16.883718 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:36:16.885033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:36:16.885033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:36:16.885033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:36:16.885033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:36:16.885033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:16.885033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:16.885033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:16.885033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:16.885033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:16.885033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:16.885033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:16.885033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:16.885033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:16.885033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:16.885033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:16.885033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:16.885954 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:16.885954 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:16.885954 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:16.885954 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:16.885954 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:36:16.885954 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:36:16.885954 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:36:16.885954 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:36:16.885954 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:36:16.885954 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:36:16.885954 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:36:16.885954 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:16.891628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 381/782] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:36:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:16.896686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b7850-c9da-4298-9414-3e8d07378e04 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:16.943229 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181af48e-5c05-4f63-952f-3e542b3eebde None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:16.996884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df0836-8c90-4efb-80f4-9f50198d60b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:16.998729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df0836-8c90-4efb-80f4-9f50198d60b0 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:16.999254 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df0836-8c90-4efb-80f4-9f50198d60b0 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:16.999754 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5df0836-8c90-4efb-80f4-9f50198d60b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:17.000754 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df0836-8c90-4efb-80f4-9f50198d60b0 None admin] RBAC: Adding query filter params (role.id=0f1c61dab22c4d97ae784468058036c7, scope.project.id=c1eae881e5714bf4bd223809e1a2d920, user.id=2b03263937cb4ca7807a24da44d37b0f) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:17.001313 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df0836-8c90-4efb-80f4-9f50198d60b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:17.003102 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5df0836-8c90-4efb-80f4-9f50198d60b0 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:17.010363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 402/783] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:36:16 2022] GET /identity/v3/role_assignments?user.id=2b03263937cb4ca7807a24da44d37b0f&scope.project.id=c1eae881e5714bf4bd223809e1a2d920&role.id=0f1c61dab22c4d97ae784468058036c7 => generated 260 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:36:17.101628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92b7850-c9da-4298-9414-3e8d07378e04 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:17.156862 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f2afe0-4df1-4bb1-9fb3-23d32f77d3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:17.157920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2afe0-4df1-4bb1-9fb3-23d32f77d3c5 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:17.158414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2afe0-4df1-4bb1-9fb3-23d32f77d3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:17.159068 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f2afe0-4df1-4bb1-9fb3-23d32f77d3c5 None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:17.160496 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f2afe0-4df1-4bb1-9fb3-23d32f77d3c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:17.161339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f2afe0-4df1-4bb1-9fb3-23d32f77d3c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:17.163752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f2afe0-4df1-4bb1-9fb3-23d32f77d3c5 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:17.181414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 382/784] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:36:16 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:17.188471 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df0836-8c90-4efb-80f4-9f50198d60b0 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:17.256049 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:17.316078 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5df0836-8c90-4efb-80f4-9f50198d60b0 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:17.354526 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b100c4-f860-42b9-9247-e6caaeb89c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:17.355622 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b100c4-f860-42b9-9247-e6caaeb89c71 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:17.356187 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b100c4-f860-42b9-9247-e6caaeb89c71 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:17.356716 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b100c4-f860-42b9-9247-e6caaeb89c71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:17.369905 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b100c4-f860-42b9-9247-e6caaeb89c71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:17.371722 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b100c4-f860-42b9-9247-e6caaeb89c71 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:17.387974 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-67b100c4-f860-42b9-9247-e6caaeb89c71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:17.387974 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:17.387974 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:17.387974 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:17.387974 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:17.387974 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:17.387974 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:17.387974 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:17.387974 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:17.387974 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:17.387974 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:17.387974 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:17.387974 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:17.387974 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:17.387974 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:17.389085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:17.389085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:17.389085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:17.389085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:17.389085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:17.389085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:17.389085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:17.389085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:17.389085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:17.389085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:17.389085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:17.389085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:17.389085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:17.389085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:17.389085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:17.389085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:17.389085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:17.390002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:17.390002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:17.390002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:17.390002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:17.392269 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 403/785] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:17.398078 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f2afe0-4df1-4bb1-9fb3-23d32f77d3c5 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:17.483616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:17.565097 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f2afe0-4df1-4bb1-9fb3-23d32f77d3c5 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:17.614067 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90c8b63-8775-409d-9a91-b02bc218f9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:17.615110 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c8b63-8775-409d-9a91-b02bc218f9f3 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:17.615596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c8b63-8775-409d-9a91-b02bc218f9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:17.616073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90c8b63-8775-409d-9a91-b02bc218f9f3 None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:17.616987 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c8b63-8775-409d-9a91-b02bc218f9f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:17.617490 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c8b63-8775-409d-9a91-b02bc218f9f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:17.619163 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90c8b63-8775-409d-9a91-b02bc218f9f3 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:17.641676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 383/786] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:17 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:17.647413 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b100c4-f860-42b9-9247-e6caaeb89c71 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:17.662183 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3d049-7da0-407b-9ce6-3d94b46be3a8 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:17.662826 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3d049-7da0-407b-9ce6-3d94b46be3a8 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:17.663337 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3d049-7da0-407b-9ce6-3d94b46be3a8 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:17.666856 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 384/787] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:17 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:17.671564 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab0a4d-8f08-46ec-9a89-20d73ca9f0d4 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:17.671844 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab0a4d-8f08-46ec-9a89-20d73ca9f0d4 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:17.672119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ab0a4d-8f08-46ec-9a89-20d73ca9f0d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:17.692917 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-c1ab0a4d-8f08-46ec-9a89-20d73ca9f0d4 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:17.694145 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-c1ab0a4d-8f08-46ec-9a89-20d73ca9f0d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:17.818739 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 385/788] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:17 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:36:17.846743 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b100c4-f860-42b9-9247-e6caaeb89c71 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:17.848396 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:17.881427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab0a4d-8f08-46ec-9a89-20d73ca9f0d4 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:17.897756 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a5105-6874-4f1f-acbb-bfae6742b7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:17.898574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a5105-6874-4f1f-acbb-bfae6742b7a0 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:17.898850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a5105-6874-4f1f-acbb-bfae6742b7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:17.899143 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7a5105-6874-4f1f-acbb-bfae6742b7a0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:17.913334 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a5105-6874-4f1f-acbb-bfae6742b7a0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:17.914784 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7a5105-6874-4f1f-acbb-bfae6742b7a0 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:17.937811 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-4b7a5105-6874-4f1f-acbb-bfae6742b7a0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:36:17.937811 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:17.937811 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:17.937811 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:17.937811 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:17.937811 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:17.937811 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:17.937811 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:17.937811 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:17.937811 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:17.937811 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:17.937811 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:17.937811 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:36:17.937811 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:36:17.937811 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:36:17.937811 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:36:17.938843 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:17.938843 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:17.938843 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:36:17.938843 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:17.938843 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:36:17.938843 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:17.938843 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:17.938843 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:17.938843 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:17.938843 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:17.938843 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:17.938843 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:17.938843 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:17.938843 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:17.938843 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:17.938843 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:17.938843 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:17.938843 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:17.940133 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:17.940133 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:17.940133 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:36:17.940133 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:36:17.940133 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:36:17.940133 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:36:17.940133 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:36:17.940133 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:36:17.940133 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:36:17.940133 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:17.940133 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 404/789] 10.222.0.115 () {56 vars in 1098 bytes} [Tue Aug 2 10:36:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:17.945562 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a5105-6874-4f1f-acbb-bfae6742b7a0 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:17.952414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:18.086847 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ab0a4d-8f08-46ec-9a89-20d73ca9f0d4 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:18.153677 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7a5105-6874-4f1f-acbb-bfae6742b7a0 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:18.164229 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2749a-5fbe-4470-adc9-e3987b98e1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:18.164879 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2749a-5fbe-4470-adc9-e3987b98e1f9 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:18.165155 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2749a-5fbe-4470-adc9-e3987b98e1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:18.165425 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d2749a-5fbe-4470-adc9-e3987b98e1f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:18.184233 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d2749a-5fbe-4470-adc9-e3987b98e1f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:36:18.190234 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-c1d2749a-5fbe-4470-adc9-e3987b98e1f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:36:18.190234 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:18.190234 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:18.190234 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:18.190234 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:18.190234 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:18.190234 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:18.190234 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:18.190234 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:18.190234 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:18.190234 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:18.190234 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:18.190234 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:36:18.190234 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:36:18.190234 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:36:18.190234 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:36:18.191366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:36:18.191366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:36:18.191366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:36:18.191366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:36:18.191366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:18.191366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:18.191366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:18.191366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:18.191366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:18.191366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:18.191366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:18.191366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:18.191366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:18.191366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:18.191366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:18.191366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:18.192704 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:18.192704 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:18.192704 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:18.192704 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:18.192704 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:36:18.192704 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:36:18.192704 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:36:18.192704 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:36:18.192704 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:36:18.192704 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:36:18.192704 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:36:18.192704 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:18.198245 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 386/790] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:36:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:36:18.202836 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2749a-5fbe-4470-adc9-e3987b98e1f9 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:18.227231 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e54a9-891a-4b03-90ad-2d6054dca478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:18.227866 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e54a9-891a-4b03-90ad-2d6054dca478 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:18.228274 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e54a9-891a-4b03-90ad-2d6054dca478 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:18.228558 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e54a9-891a-4b03-90ad-2d6054dca478 None admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:18.229540 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e54a9-891a-4b03-90ad-2d6054dca478 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:18.229857 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e54a9-891a-4b03-90ad-2d6054dca478 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:18.231625 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e54a9-891a-4b03-90ad-2d6054dca478 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:18.251924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 405/791] 10.222.0.115 () {56 vars in 1138 bytes} [Tue Aug 2 10:36:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:36:18.253294 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ab8c0-370f-41df-bf45-90a53da9627c None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:18.253587 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ab8c0-370f-41df-bf45-90a53da9627c None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:18.253866 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ab8c0-370f-41df-bf45-90a53da9627c None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:18.259761 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 406/792] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:18 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:18.260638 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ab8c0-370f-41df-bf45-90a53da9627c None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:18.307457 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:18.317790 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:18.327213 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d2749a-5fbe-4470-adc9-e3987b98e1f9 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:18.365980 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3750fc-8de2-4903-b4aa-05b3908d3556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:18.366520 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3750fc-8de2-4903-b4aa-05b3908d3556 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:18.366854 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3750fc-8de2-4903-b4aa-05b3908d3556 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:18.367157 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3750fc-8de2-4903-b4aa-05b3908d3556 None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:18.367885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3750fc-8de2-4903-b4aa-05b3908d3556 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:18.368188 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3750fc-8de2-4903-b4aa-05b3908d3556 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:18.369570 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3750fc-8de2-4903-b4aa-05b3908d3556 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:18.384379 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 387/793] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:36:18 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:18.384379 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ac92c-e694-4c82-9ada-5e6d663935b1 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:18.384379 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ac92c-e694-4c82-9ada-5e6d663935b1 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:18.384379 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0ac92c-e694-4c82-9ada-5e6d663935b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:18.402249 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-9c0ac92c-e694-4c82-9ada-5e6d663935b1 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:18.403597 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-9c0ac92c-e694-4c82-9ada-5e6d663935b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:18.409195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ab8c0-370f-41df-bf45-90a53da9627c None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:18.451584 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d52a31-e0b4-48ca-bd4a-d991a6cfb332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:18.452824 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d52a31-e0b4-48ca-bd4a-d991a6cfb332 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:18.453436 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d52a31-e0b4-48ca-bd4a-d991a6cfb332 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:18.454082 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d52a31-e0b4-48ca-bd4a-d991a6cfb332 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:18.486920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 388/794] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:18 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:18.487888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ac92c-e694-4c82-9ada-5e6d663935b1 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:18.561939 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d52a31-e0b4-48ca-bd4a-d991a6cfb332 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:18.563899 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d52a31-e0b4-48ca-bd4a-d991a6cfb332 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:18.590650 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0ac92c-e694-4c82-9ada-5e6d663935b1 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:18.630372 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888d0321-4858-4770-818f-6eba61e2b0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:18.630372 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d0321-4858-4770-818f-6eba61e2b0d3 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:18.630682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:18.630780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d0321-4858-4770-818f-6eba61e2b0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:18.630780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888d0321-4858-4770-818f-6eba61e2b0d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:18.647436 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888d0321-4858-4770-818f-6eba61e2b0d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:18.649256 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-888d0321-4858-4770-818f-6eba61e2b0d3 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:18.671880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-888d0321-4858-4770-818f-6eba61e2b0d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:18.671880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:18.671880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:18.671880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:18.671880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:18.671880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:18.671880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:18.671880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:18.671880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:18.671880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:18.671880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:18.671880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:18.671880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:18.671880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:18.671880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:18.673173 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:18.673173 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:18.673173 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:18.673173 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:18.673173 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:18.673173 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:18.673173 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:18.673173 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:18.673173 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:18.673173 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:18.673173 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:18.673173 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:18.673173 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:18.673173 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:18.673173 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:18.673173 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:18.673173 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:18.674323 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:18.674323 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:18.674323 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:18.674323 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:18.674323 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 389/795] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:18.674323 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888d0321-4858-4770-818f-6eba61e2b0d3 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:18.681130 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:18.691076 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 407/796] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:36:18 2022] GET /identity/v3/auth/tokens => generated 1003 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:36:18.692351 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d52a31-e0b4-48ca-bd4a-d991a6cfb332 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:18.810969 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d52a31-e0b4-48ca-bd4a-d991a6cfb332 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:18.817595 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888d0321-4858-4770-818f-6eba61e2b0d3 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:18.836345 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:18.858792 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b6988-4fa3-4293-84c6-d97328466b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:18.859721 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b6988-4fa3-4293-84c6-d97328466b18 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:18.860006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b6988-4fa3-4293-84c6-d97328466b18 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:18.860283 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b6988-4fa3-4293-84c6-d97328466b18 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:18.860996 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b6988-4fa3-4293-84c6-d97328466b18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:18.861300 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b6988-4fa3-4293-84c6-d97328466b18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:18.862135 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea7610b-9e63-4749-bafc-5cb32dc946f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:18.862681 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b6988-4fa3-4293-84c6-d97328466b18 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:18.863331 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7610b-9e63-4749-bafc-5cb32dc946f2 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:18.863823 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7610b-9e63-4749-bafc-5cb32dc946f2 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:18.864356 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7610b-9e63-4749-bafc-5cb32dc946f2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:18.865192 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea7610b-9e63-4749-bafc-5cb32dc946f2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:18.866843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea7610b-9e63-4749-bafc-5cb32dc946f2 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:18.877499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-0ea7610b-9e63-4749-bafc-5cb32dc946f2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 10:36:18.877499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:18.877499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:36:18.877499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:36:18.877499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:36:18.877499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:36:18.877499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:36:18.877499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:18.877499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 10:36:18.877499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:18.877499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:36:18.877499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:18.877499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:18.877499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:18.877499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:18.877499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:18.878561 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:18.878561 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:18.878561 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:18.878561 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:18.878561 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:18.878561 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:18.878561 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:18.878561 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:36:18.878561 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:36:18.878561 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:36:18.878561 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:36:18.878561 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:18.878561 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:18.878561 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:18.878561 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:18.878561 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:18.879853 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:18.879853 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:18.879853 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:18.879853 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:18.879853 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:18.879853 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:18.879853 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:18.879853 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:18.879853 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:18.879853 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:18.879853 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:18.879853 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:36:18.879853 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:18.879853 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 02 10:36:18.879853 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:18.882568 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 390/797] 10.222.0.115 () {56 vars in 1134 bytes} [Tue Aug 2 10:36:18 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:18.883219 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 408/798] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:18 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:18.884129 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b6988-4fa3-4293-84c6-d97328466b18 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:18.884349 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea7610b-9e63-4749-bafc-5cb32dc946f2 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:18.885119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:19.017541 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b6988-4fa3-4293-84c6-d97328466b18 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:19.018000 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea7610b-9e63-4749-bafc-5cb32dc946f2 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:19.063330 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b76747-ea20-43ca-8604-594dceafe35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:19.064038 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b76747-ea20-43ca-8604-594dceafe35a None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:19.064374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b76747-ea20-43ca-8604-594dceafe35a None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:19.064649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b76747-ea20-43ca-8604-594dceafe35a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:19.069136 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06a03ac-f820-4981-a4b7-4ab793fcb81b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:19.070069 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a03ac-f820-4981-a4b7-4ab793fcb81b None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:19.070561 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a03ac-f820-4981-a4b7-4ab793fcb81b None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:19.071037 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a03ac-f820-4981-a4b7-4ab793fcb81b None admin] PATH_INFO: `/v3/users/swift` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:19.077740 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b76747-ea20-43ca-8604-594dceafe35a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:19.079224 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b76747-ea20-43ca-8604-594dceafe35a None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:19.080893 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a03ac-f820-4981-a4b7-4ab793fcb81b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:19.082311 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06a03ac-f820-4981-a4b7-4ab793fcb81b None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:19.097400 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-a06a03ac-f820-4981-a4b7-4ab793fcb81b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:36:19.097400 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:19.097400 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:19.097400 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:19.097400 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:19.097400 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:19.097400 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:19.097400 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:19.097400 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:19.097400 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:19.097400 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:19.097400 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:19.097400 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:36:19.097400 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:36:19.097400 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:36:19.097400 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:36:19.098476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:19.098476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:19.098476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:36:19.098476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:19.098476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:36:19.098476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:19.098476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:19.098476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:19.098476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:19.098476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:19.098476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:19.098476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:19.098476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:19.098476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:19.098476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:19.098476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:19.098476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:19.098476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:19.099846 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:19.099846 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:19.099846 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:36:19.099846 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:36:19.099846 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:36:19.099846 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:36:19.099846 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:36:19.099846 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:36:19.099846 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:36:19.099846 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:19.100525 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-a7b76747-ea20-43ca-8604-594dceafe35a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:19.100525 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:19.100525 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:19.100525 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:19.100525 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:19.100525 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:19.100525 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:19.100525 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:19.100525 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:19.100525 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:19.100525 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:19.100525 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:19.100525 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:19.100525 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:19.100525 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:19.101334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:19.101334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:19.101334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:19.101334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:19.101334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:19.101334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:19.101334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:19.101334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:19.101334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:19.101334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:19.101334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:19.101334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:19.101334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:19.101334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:19.101334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:19.101334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:19.101334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:19.104146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:19.104146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:19.104146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:19.104146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:19.104146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 391/799] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:19.104146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b76747-ea20-43ca-8604-594dceafe35a None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:19.109904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 409/800] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:36:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:19.109904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5824cf4-1f3a-4eef-a663-b1dd419af512 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:19.109904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5824cf4-1f3a-4eef-a663-b1dd419af512 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:19.109904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5824cf4-1f3a-4eef-a663-b1dd419af512 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:19.109904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 410/801] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:19 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:19.111003 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5824cf4-1f3a-4eef-a663-b1dd419af512 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:19.176170 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:19.180987 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:19.250292 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b76747-ea20-43ca-8604-594dceafe35a None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:19.255214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5824cf4-1f3a-4eef-a663-b1dd419af512 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:19.290842 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd5838-ef94-4103-8e8d-b7b1a226b661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:19.291700 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd5838-ef94-4103-8e8d-b7b1a226b661 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:19.292400 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd5838-ef94-4103-8e8d-b7b1a226b661 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:19.292680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20dd5838-ef94-4103-8e8d-b7b1a226b661 None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:19.293362 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd5838-ef94-4103-8e8d-b7b1a226b661 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:19.293665 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd5838-ef94-4103-8e8d-b7b1a226b661 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:19.295679 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20dd5838-ef94-4103-8e8d-b7b1a226b661 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:19.306331 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 392/802] 10.222.0.115 () {56 vars in 1129 bytes} [Tue Aug 2 10:36:19 2022] GET /identity/v3/services?name=block-storage => generated 131 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:19.306331 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd5838-ef94-4103-8e8d-b7b1a226b661 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:19.308633 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfeb1dd4-91f6-4a07-9c0d-eebf7b70bea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:19.309353 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb1dd4-91f6-4a07-9c0d-eebf7b70bea4 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:19.309812 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb1dd4-91f6-4a07-9c0d-eebf7b70bea4 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:19.309979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfeb1dd4-91f6-4a07-9c0d-eebf7b70bea4 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:19.311006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeb1dd4-91f6-4a07-9c0d-eebf7b70bea4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:19.311581 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeb1dd4-91f6-4a07-9c0d-eebf7b70bea4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:19.313484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfeb1dd4-91f6-4a07-9c0d-eebf7b70bea4 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:19.332771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 411/803] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:19 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:19.334354 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d987c75-2e0c-4258-a964-e0b990a21885 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:19.334984 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d987c75-2e0c-4258-a964-e0b990a21885 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:19.334984 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d987c75-2e0c-4258-a964-e0b990a21885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:19.359406 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-3d987c75-2e0c-4258-a964-e0b990a21885 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:19.360754 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-3d987c75-2e0c-4258-a964-e0b990a21885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:19.446902 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20dd5838-ef94-4103-8e8d-b7b1a226b661 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:19.468260 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:19.498131 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 412/804] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:19 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:19.498354 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fedb646-fdcb-4c0d-8088-daf0d1e088d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:19.498354 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fedb646-fdcb-4c0d-8088-daf0d1e088d4 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:19.498354 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fedb646-fdcb-4c0d-8088-daf0d1e088d4 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:19.498607 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d987c75-2e0c-4258-a964-e0b990a21885 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:19.498680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fedb646-fdcb-4c0d-8088-daf0d1e088d4 None admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:19.498680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fedb646-fdcb-4c0d-8088-daf0d1e088d4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:19.498680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fedb646-fdcb-4c0d-8088-daf0d1e088d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:19.498680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fedb646-fdcb-4c0d-8088-daf0d1e088d4 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:19.520452 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 393/805] 10.222.0.115 () {56 vars in 1140 bytes} [Tue Aug 2 10:36:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:19.521516 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fedb646-fdcb-4c0d-8088-daf0d1e088d4 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:19.549454 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:19.644554 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d987c75-2e0c-4258-a964-e0b990a21885 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:19.670630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fedb646-fdcb-4c0d-8088-daf0d1e088d4 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:19.690764 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10101a-0aae-4b9e-8bac-2bccbb39f134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:19.691419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10101a-0aae-4b9e-8bac-2bccbb39f134 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:19.691697 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10101a-0aae-4b9e-8bac-2bccbb39f134 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:19.691986 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10101a-0aae-4b9e-8bac-2bccbb39f134 None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:19.692667 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e10101a-0aae-4b9e-8bac-2bccbb39f134 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:19.692972 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e10101a-0aae-4b9e-8bac-2bccbb39f134 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:19.694337 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e10101a-0aae-4b9e-8bac-2bccbb39f134 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:19.699796 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 413/806] 10.222.0.115 () {56 vars in 1129 bytes} [Tue Aug 2 10:36:19 2022] GET /identity/v3/services?type=block-storage => generated 369 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:19.700785 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10101a-0aae-4b9e-8bac-2bccbb39f134 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:19.713005 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c069757-4008-42b6-b2e6-911758b23c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:19.713649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c069757-4008-42b6-b2e6-911758b23c64 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:19.713935 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c069757-4008-42b6-b2e6-911758b23c64 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:19.714212 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c069757-4008-42b6-b2e6-911758b23c64 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:19.721716 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c069757-4008-42b6-b2e6-911758b23c64 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:19.723189 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c069757-4008-42b6-b2e6-911758b23c64 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:19.735339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-9c069757-4008-42b6-b2e6-911758b23c64 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:19.735339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:19.735339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:19.735339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:19.735339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:19.735339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:19.735339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:19.735339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:19.735339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:19.735339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:19.735339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:19.735339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:19.735339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:36:19.735339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:36:19.735339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:36:19.736472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:36:19.736472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:19.736472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:19.736472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:19.736472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:19.736472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:19.736472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:19.736472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:19.736472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:19.736472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:19.736472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:19.736472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:19.736472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:19.736472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:19.736472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:19.736472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:19.736472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:19.737424 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:36:19.737424 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:36:19.737424 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:36:19.737424 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:36:19.737424 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:36:19.737424 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:36:19.737424 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:19.737424 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:19.737424 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 394/807] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:36:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:19.737424 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c069757-4008-42b6-b2e6-911758b23c64 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:19.771945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:19.813730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:19.821122 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10101a-0aae-4b9e-8bac-2bccbb39f134 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:19.858888 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b17abf-8942-4ce5-9eb5-0d74a221571c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:19.859687 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b17abf-8942-4ce5-9eb5-0d74a221571c None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:19.860051 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b17abf-8942-4ce5-9eb5-0d74a221571c None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:19.860564 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b17abf-8942-4ce5-9eb5-0d74a221571c None admin] PATH_INFO: `/v3/services/network` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:19.861489 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b17abf-8942-4ce5-9eb5-0d74a221571c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:19.863081 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b17abf-8942-4ce5-9eb5-0d74a221571c None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:19.873403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-59b17abf-8942-4ce5-9eb5-0d74a221571c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 10:36:19.873403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:19.873403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:36:19.873403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:36:19.873403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:36:19.873403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:36:19.873403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:36:19.873403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:19.873403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 10:36:19.873403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:19.873403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:36:19.873403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:19.873403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:19.873403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:19.873403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:19.873403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:19.873403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:19.876356 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:19.876356 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:19.876356 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:19.876356 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:19.876356 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:19.876356 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:19.876356 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:36:19.876356 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:36:19.876356 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:36:19.876356 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:36:19.876356 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:19.876356 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:19.876356 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:19.876356 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:19.876356 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:19.876356 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:19.879949 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:19.879949 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:19.879949 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:19.879949 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:19.879949 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:19.879949 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:19.879949 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:19.879949 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:19.879949 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:19.879949 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:19.879949 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:36:19.879949 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:19.879949 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 02 10:36:19.879949 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:19.882410 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c069757-4008-42b6-b2e6-911758b23c64 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:19.882568 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 414/808] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:36:19 2022] GET /identity/v3/services/network => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:19.882568 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f10ec6-cddf-41c8-be7c-dbff5a619512 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:19.882568 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f10ec6-cddf-41c8-be7c-dbff5a619512 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:19.882568 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f10ec6-cddf-41c8-be7c-dbff5a619512 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:19.882568 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 415/809] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:19 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:19.884026 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f10ec6-cddf-41c8-be7c-dbff5a619512 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:19.917688 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab4456-a40d-402f-ab30-04d739f7c379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:19.918327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab4456-a40d-402f-ab30-04d739f7c379 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:19.918606 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab4456-a40d-402f-ab30-04d739f7c379 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:19.918874 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ab4456-a40d-402f-ab30-04d739f7c379 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:19.929869 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab4456-a40d-402f-ab30-04d739f7c379 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:19.931341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ab4456-a40d-402f-ab30-04d739f7c379 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:19.949337 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-12ab4456-a40d-402f-ab30-04d739f7c379 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:19.949337 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:19.949337 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:19.949337 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:19.949337 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:19.949337 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:19.949337 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:19.949337 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:19.949337 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:19.949337 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:19.949337 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:19.949337 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:19.949337 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:19.949337 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:19.949337 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:19.950510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:19.950510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:19.950510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:19.950510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:19.950510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:19.950510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:19.950510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:19.950510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:19.950510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:19.950510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:19.950510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:19.950510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:19.950510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:19.950510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:19.950510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:19.950510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:19.950510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:19.951839 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:19.951839 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:19.951839 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:19.951839 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:19.951839 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 395/810] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:19.951839 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab4456-a40d-402f-ab30-04d739f7c379 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:20.002167 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f10ec6-cddf-41c8-be7c-dbff5a619512 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:20.042923 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15a4044-5897-4d20-be98-89f3378a01a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:20.043346 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:20.043565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a4044-5897-4d20-be98-89f3378a01a1 None admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:20.043841 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a4044-5897-4d20-be98-89f3378a01a1 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:20.044109 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15a4044-5897-4d20-be98-89f3378a01a1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:20.045482 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a4044-5897-4d20-be98-89f3378a01a1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:20.046680 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15a4044-5897-4d20-be98-89f3378a01a1 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:20.059948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ab4456-a40d-402f-ab30-04d739f7c379 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:20.099700 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac94b4-f402-48a9-ae3d-e8ea6c612a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:20.100314 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac94b4-f402-48a9-ae3d-e8ea6c612a46 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:20.100586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac94b4-f402-48a9-ae3d-e8ea6c612a46 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:20.100853 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac94b4-f402-48a9-ae3d-e8ea6c612a46 None admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:20.101593 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac94b4-f402-48a9-ae3d-e8ea6c612a46 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:20.101936 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac94b4-f402-48a9-ae3d-e8ea6c612a46 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:20.103371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac94b4-f402-48a9-ae3d-e8ea6c612a46 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:20.116107 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 396/811] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:36:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:20.117457 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac94b4-f402-48a9-ae3d-e8ea6c612a46 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:20.170359 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 416/812] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:36:19 2022] POST /identity/v3/endpoints => generated 348 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:36:20.171782 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a30c5a-e9d3-4a25-9eb1-87b32f677c2e None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:20.172084 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a30c5a-e9d3-4a25-9eb1-87b32f677c2e None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:20.172366 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a30c5a-e9d3-4a25-9eb1-87b32f677c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:20.203080 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-58a30c5a-e9d3-4a25-9eb1-87b32f677c2e None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:20.205697 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-58a30c5a-e9d3-4a25-9eb1-87b32f677c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:20.219994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:20.236054 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac94b4-f402-48a9-ae3d-e8ea6c612a46 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:20.284580 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235443c2-816f-4f59-a417-bf3fb1873fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:20.285215 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235443c2-816f-4f59-a417-bf3fb1873fea None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:20.285509 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235443c2-816f-4f59-a417-bf3fb1873fea None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:20.285778 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235443c2-816f-4f59-a417-bf3fb1873fea None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:20.286473 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235443c2-816f-4f59-a417-bf3fb1873fea None admin] RBAC: Adding query filter params (name=network) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:20.286777 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235443c2-816f-4f59-a417-bf3fb1873fea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:20.288447 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235443c2-816f-4f59-a417-bf3fb1873fea None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:20.293486 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 397/813] 10.222.0.115 () {56 vars in 1117 bytes} [Tue Aug 2 10:36:20 2022] GET /identity/v3/services?name=network => generated 125 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:20.294414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235443c2-816f-4f59-a417-bf3fb1873fea None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:20.297655 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 417/814] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:20 2022] POST /identity/v3/auth/tokens => generated 1003 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:20.299308 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a30c5a-e9d3-4a25-9eb1-87b32f677c2e None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:20.308734 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:20.428207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a30c5a-e9d3-4a25-9eb1-87b32f677c2e None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:20.431276 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235443c2-816f-4f59-a417-bf3fb1873fea None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:20.481787 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffdd21-7fcf-4375-b713-ca07312470c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:20.483346 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffdd21-7fcf-4375-b713-ca07312470c6 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:20.483699 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffdd21-7fcf-4375-b713-ca07312470c6 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:20.483933 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fffdd21-7fcf-4375-b713-ca07312470c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:20.484715 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fffdd21-7fcf-4375-b713-ca07312470c6 None admin] RBAC: Adding query filter params (role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac, scope.project.id=c1eae881e5714bf4bd223809e1a2d920, user.id=d0478bd540f446ec810e2091c46ac49c) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:20.484844 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a682ac-c030-418d-8877-b6dfdac9f59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:20.485283 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fffdd21-7fcf-4375-b713-ca07312470c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:20.485478 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a682ac-c030-418d-8877-b6dfdac9f59e None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:20.485719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a682ac-c030-418d-8877-b6dfdac9f59e None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:20.486057 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a682ac-c030-418d-8877-b6dfdac9f59e None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:20.486472 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fffdd21-7fcf-4375-b713-ca07312470c6 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:20.486747 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a682ac-c030-418d-8877-b6dfdac9f59e None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:20.487061 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a682ac-c030-418d-8877-b6dfdac9f59e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:20.488618 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a682ac-c030-418d-8877-b6dfdac9f59e None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:20.509120 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 418/815] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:36:20 2022] GET /identity/v3/role_assignments?user.id=d0478bd540f446ec810e2091c46ac49c&scope.project.id=c1eae881e5714bf4bd223809e1a2d920&role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 610 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:20.509120 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffdd21-7fcf-4375-b713-ca07312470c6 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:20.509416 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 398/816] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:20 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:20.509416 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a682ac-c030-418d-8877-b6dfdac9f59e None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:20.522778 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:20.596545 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:20.652541 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a682ac-c030-418d-8877-b6dfdac9f59e None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:20.654567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fffdd21-7fcf-4375-b713-ca07312470c6 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:20.696479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89753e-f799-424c-b9b2-5f4d616da403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:20.697283 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89753e-f799-424c-b9b2-5f4d616da403 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:20.699194 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89753e-f799-424c-b9b2-5f4d616da403 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:20.699194 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e89753e-f799-424c-b9b2-5f4d616da403 None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:20.699194 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e89753e-f799-424c-b9b2-5f4d616da403 None admin] RBAC: Adding query filter params (type=network) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:20.699566 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e89753e-f799-424c-b9b2-5f4d616da403 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:20.700775 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50002c13-1238-4b06-9647-2b3359c51460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:20.701048 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e89753e-f799-424c-b9b2-5f4d616da403 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:20.702483 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50002c13-1238-4b06-9647-2b3359c51460 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:20.703208 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50002c13-1238-4b06-9647-2b3359c51460 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:20.704039 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50002c13-1238-4b06-9647-2b3359c51460 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:20.708104 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 419/817] 10.222.0.115 () {56 vars in 1117 bytes} [Tue Aug 2 10:36:20 2022] GET /identity/v3/services?type=network => generated 352 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:20.709766 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89753e-f799-424c-b9b2-5f4d616da403 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:20.724924 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50002c13-1238-4b06-9647-2b3359c51460 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:36:20.731676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-50002c13-1238-4b06-9647-2b3359c51460 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:36:20.731676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:20.731676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:20.731676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:20.731676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:20.731676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:20.731676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:20.731676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:20.731676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:20.731676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:20.731676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:20.731676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:20.731676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:36:20.731676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:36:20.731676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:36:20.731676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:36:20.733105 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:36:20.733105 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:36:20.733105 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:36:20.733105 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:36:20.733105 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:20.733105 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:20.733105 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:20.733105 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:20.733105 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:20.733105 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:20.733105 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:20.733105 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:20.733105 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:20.733105 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:20.733105 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:20.733105 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:20.734033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:20.734033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:20.734033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:20.734033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:20.734033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:36:20.734033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:36:20.734033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:36:20.734033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:36:20.734033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:36:20.734033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:36:20.734033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:36:20.734033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:20.734033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 399/818] 10.222.0.115 () {56 vars in 1107 bytes} [Tue Aug 2 10:36:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:20.734033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50002c13-1238-4b06-9647-2b3359c51460 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:20.832463 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:20.875467 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50002c13-1238-4b06-9647-2b3359c51460 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:20.883539 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e89753e-f799-424c-b9b2-5f4d616da403 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:20.910749 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:20.916213 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80611969-70b0-4885-8db0-60cbd58cb8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:20.916992 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80611969-70b0-4885-8db0-60cbd58cb8f4 None admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:20.917275 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80611969-70b0-4885-8db0-60cbd58cb8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:20.917838 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80611969-70b0-4885-8db0-60cbd58cb8f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:20.918820 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80611969-70b0-4885-8db0-60cbd58cb8f4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:20.922280 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80611969-70b0-4885-8db0-60cbd58cb8f4 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:20.936217 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ebb57e-18c3-4528-98f4-f40d173bc806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:20.937019 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ebb57e-18c3-4528-98f4-f40d173bc806 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:20.937379 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ebb57e-18c3-4528-98f4-f40d173bc806 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:20.937766 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ebb57e-18c3-4528-98f4-f40d173bc806 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:20.946216 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ebb57e-18c3-4528-98f4-f40d173bc806 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:20.947869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ebb57e-18c3-4528-98f4-f40d173bc806 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:20.973171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-98ebb57e-18c3-4528-98f4-f40d173bc806 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:20.973171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:20.973171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:20.973171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:20.973171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:20.973171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:20.973171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:20.973171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:20.973171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:20.973171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:20.973171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:20.973171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:20.973171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:36:20.973171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:36:20.973171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:36:20.974153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:36:20.974153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:20.974153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:20.974153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:20.974153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:20.974153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:20.974153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:20.974153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:20.974153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:20.974153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:20.974153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:20.974153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:20.974153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:20.974153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:20.974153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:20.974153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:20.974153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:20.975066 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:36:20.975066 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:36:20.975066 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:36:20.975066 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:36:20.975066 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:36:20.975066 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:36:20.975066 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:20.975066 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:20.975066 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 420/819] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:36:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:20.975066 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ebb57e-18c3-4528-98f4-f40d173bc806 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:21.065852 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 400/820] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:36:20 2022] POST /identity/v3/endpoints => generated 339 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:36:21.071701 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80611969-70b0-4885-8db0-60cbd58cb8f4 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:21.129227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ebb57e-18c3-4528-98f4-f40d173bc806 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:21.171526 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:21.198380 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b6cca-6f73-4280-af29-dcf032827aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:21.202718 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b6cca-6f73-4280-af29-dcf032827aaa None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:21.203037 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b6cca-6f73-4280-af29-dcf032827aaa None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:21.203386 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b6cca-6f73-4280-af29-dcf032827aaa None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:21.204201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b6cca-6f73-4280-af29-dcf032827aaa None admin] RBAC: Adding query filter params (name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:21.204471 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b6cca-6f73-4280-af29-dcf032827aaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:21.206127 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b6cca-6f73-4280-af29-dcf032827aaa None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:21.222684 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 421/821] 10.222.0.115 () {56 vars in 1108 bytes} [Tue Aug 2 10:36:20 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:21.230994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b6cca-6f73-4280-af29-dcf032827aaa None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:21.315740 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80611969-70b0-4885-8db0-60cbd58cb8f4 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:21.360441 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48debc9d-2b3d-4b31-b603-990c5ce804ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:21.361467 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48debc9d-2b3d-4b31-b603-990c5ce804ff None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:21.361952 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48debc9d-2b3d-4b31-b603-990c5ce804ff None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:21.362438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48debc9d-2b3d-4b31-b603-990c5ce804ff None admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:21.363563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48debc9d-2b3d-4b31-b603-990c5ce804ff None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:21.364081 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48debc9d-2b3d-4b31-b603-990c5ce804ff None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:21.365749 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48debc9d-2b3d-4b31-b603-990c5ce804ff None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:21.380162 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 401/822] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:36:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:21.386568 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48debc9d-2b3d-4b31-b603-990c5ce804ff None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:21.414682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b6cca-6f73-4280-af29-dcf032827aaa None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:21.441564 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:21.461756 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3693ee5-43b0-42c8-ba8d-1292e61a654c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:21.462693 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3693ee5-43b0-42c8-ba8d-1292e61a654c None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:21.463176 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3693ee5-43b0-42c8-ba8d-1292e61a654c None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:21.463651 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3693ee5-43b0-42c8-ba8d-1292e61a654c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:21.477725 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3693ee5-43b0-42c8-ba8d-1292e61a654c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:21.483180 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3693ee5-43b0-42c8-ba8d-1292e61a654c None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:21.504710 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-a3693ee5-43b0-42c8-ba8d-1292e61a654c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:21.504710 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:21.504710 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:21.504710 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:21.504710 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:21.504710 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:21.504710 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:21.504710 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:21.504710 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:21.504710 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:21.504710 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:21.504710 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:21.504710 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:21.504710 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:21.504710 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:21.505716 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:21.505716 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:21.505716 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:21.505716 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:21.505716 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:21.505716 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:21.505716 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:21.505716 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:21.505716 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:21.505716 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:21.505716 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:21.505716 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:21.505716 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:21.505716 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:21.505716 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:21.505716 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:21.505716 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:21.506660 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:21.506660 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:21.506660 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:21.506660 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:21.508945 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 422/823] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:21.517312 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3693ee5-43b0-42c8-ba8d-1292e61a654c None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:21.522576 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48debc9d-2b3d-4b31-b603-990c5ce804ff None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:21.564855 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b435977-f9bf-45b8-89a8-fb6253467697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:21.565910 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b435977-f9bf-45b8-89a8-fb6253467697 None admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:21.566413 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b435977-f9bf-45b8-89a8-fb6253467697 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:21.566906 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b435977-f9bf-45b8-89a8-fb6253467697 None admin] PATH_INFO: `/v3/projects/c1eae881e5714bf4bd223809e1a2d920/users/0daa49b5a35f4a468bb8d0939e7ec20c/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:21.569316 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b435977-f9bf-45b8-89a8-fb6253467697 None admin] RBAC: Authorizing `identity:create_grant(project_id=c1eae881e5714bf4bd223809e1a2d920, user_id=0daa49b5a35f4a468bb8d0939e7ec20c, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:21.571026 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b435977-f9bf-45b8-89a8-fb6253467697 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:21.577905 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:21.649572 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3693ee5-43b0-42c8-ba8d-1292e61a654c None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:21.679064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 402/824] 10.222.0.115 () {58 vars in 1444 bytes} [Tue Aug 2 10:36:21 2022] PUT /identity/v3/projects/c1eae881e5714bf4bd223809e1a2d920/users/0daa49b5a35f4a468bb8d0939e7ec20c/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:36:21.705073 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414c754-c12e-485a-a08f-b1407bfb7b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:21.705864 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414c754-c12e-485a-a08f-b1407bfb7b73 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:21.706188 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414c754-c12e-485a-a08f-b1407bfb7b73 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:21.706438 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414c754-c12e-485a-a08f-b1407bfb7b73 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:21.707267 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414c754-c12e-485a-a08f-b1407bfb7b73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:21.707549 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414c754-c12e-485a-a08f-b1407bfb7b73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:21.709423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414c754-c12e-485a-a08f-b1407bfb7b73 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:21.726604 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 423/825] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:21 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:21.732964 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b435977-f9bf-45b8-89a8-fb6253467697 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:21.866502 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:21.869968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b435977-f9bf-45b8-89a8-fb6253467697 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:21.916025 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f95d3-a7ea-49e4-81fd-1a5ecfe76326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:21.916741 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f95d3-a7ea-49e4-81fd-1a5ecfe76326 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:21.917018 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f95d3-a7ea-49e4-81fd-1a5ecfe76326 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:21.917289 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669f95d3-a7ea-49e4-81fd-1a5ecfe76326 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:21.926226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f95d3-a7ea-49e4-81fd-1a5ecfe76326 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:21.927766 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669f95d3-a7ea-49e4-81fd-1a5ecfe76326 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:21.941386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-669f95d3-a7ea-49e4-81fd-1a5ecfe76326 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:36:21.941386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:21.941386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:21.941386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:21.941386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:21.941386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:21.941386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:21.941386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:21.941386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:21.941386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:21.941386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:21.941386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:21.941386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:36:21.941386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:36:21.941386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:36:21.941386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:36:21.942397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:21.942397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:21.942397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:36:21.942397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:21.942397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:36:21.942397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:21.942397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:21.942397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:21.942397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:21.942397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:21.942397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:21.942397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:21.942397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:21.942397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:21.942397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:21.942397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:21.942397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:21.942397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:21.943414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:21.943414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:21.943414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:36:21.943414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:36:21.943414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:36:21.943414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:36:21.943414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:36:21.943414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:36:21.943414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:36:21.943414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:21.947359 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 403/826] 10.222.0.115 () {56 vars in 1122 bytes} [Tue Aug 2 10:36:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:36:21.948558 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414c754-c12e-485a-a08f-b1407bfb7b73 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:22.137283 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414c754-c12e-485a-a08f-b1407bfb7b73 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:22.139845 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:22.201704 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32be90da-76ba-4482-a8ef-7a5054dfd9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:22.202729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be90da-76ba-4482-a8ef-7a5054dfd9da None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:22.203253 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be90da-76ba-4482-a8ef-7a5054dfd9da None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:22.203724 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32be90da-76ba-4482-a8ef-7a5054dfd9da None admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:22.205394 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32be90da-76ba-4482-a8ef-7a5054dfd9da None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:22.205891 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32be90da-76ba-4482-a8ef-7a5054dfd9da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:22.208370 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32be90da-76ba-4482-a8ef-7a5054dfd9da None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:22.238278 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 424/827] 10.222.0.115 () {56 vars in 1154 bytes} [Tue Aug 2 10:36:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:22.244532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f95d3-a7ea-49e4-81fd-1a5ecfe76326 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:22.364509 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669f95d3-a7ea-49e4-81fd-1a5ecfe76326 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:22.410116 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b556e58-9e07-436f-b0ab-4a6ad520391b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:22.411677 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b556e58-9e07-436f-b0ab-4a6ad520391b None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:22.412794 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b556e58-9e07-436f-b0ab-4a6ad520391b None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:22.413386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b556e58-9e07-436f-b0ab-4a6ad520391b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:22.428239 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b556e58-9e07-436f-b0ab-4a6ad520391b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:22.430031 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b556e58-9e07-436f-b0ab-4a6ad520391b None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:22.446644 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-1b556e58-9e07-436f-b0ab-4a6ad520391b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:22.446644 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:22.446644 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:22.446644 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:22.446644 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:22.446644 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:22.446644 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:22.446644 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:22.446644 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:22.446644 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:22.446644 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:22.446644 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:22.446644 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:22.446644 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:22.446644 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:22.447722 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:22.447722 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:22.447722 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:22.447722 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:22.447722 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:22.447722 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:22.447722 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:22.447722 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:22.447722 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:22.447722 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:22.447722 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:22.447722 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:22.447722 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:22.447722 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:22.447722 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:22.447722 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:22.447722 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:22.448967 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:22.448967 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:22.448967 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:22.448967 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:22.451976 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 404/828] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:36:22.453763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5969c4-8d8f-4333-824d-01af87c2a838 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:22.454064 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5969c4-8d8f-4333-824d-01af87c2a838 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:22.454347 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5969c4-8d8f-4333-824d-01af87c2a838 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:22.459079 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 425/829] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:22.459734 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b556e58-9e07-436f-b0ab-4a6ad520391b None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:22.461771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635d9e1-0da3-4ecc-b30b-2344c7bb4ec4 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:22.462059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635d9e1-0da3-4ecc-b30b-2344c7bb4ec4 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:22.462480 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4635d9e1-0da3-4ecc-b30b-2344c7bb4ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:22.478117 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:22.485829 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-4635d9e1-0da3-4ecc-b30b-2344c7bb4ec4 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:22.491915 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-4635d9e1-0da3-4ecc-b30b-2344c7bb4ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:22.589753 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b556e58-9e07-436f-b0ab-4a6ad520391b None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:22.611534 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 426/830] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:22 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:22.627763 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9358462-2793-405d-a2fd-724bbe0d1516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:22.628383 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9358462-2793-405d-a2fd-724bbe0d1516 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:22.628652 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9358462-2793-405d-a2fd-724bbe0d1516 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:22.628923 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9358462-2793-405d-a2fd-724bbe0d1516 None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:22.629618 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9358462-2793-405d-a2fd-724bbe0d1516 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:22.629917 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9358462-2793-405d-a2fd-724bbe0d1516 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:22.631453 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9358462-2793-405d-a2fd-724bbe0d1516 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:22.639400 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635d9e1-0da3-4ecc-b30b-2344c7bb4ec4 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:22.645644 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 405/831] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:22.650790 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9358462-2793-405d-a2fd-724bbe0d1516 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:22.731498 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:22.740779 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:22.751530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4635d9e1-0da3-4ecc-b30b-2344c7bb4ec4 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:22.763384 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9358462-2793-405d-a2fd-724bbe0d1516 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:22.789885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b452d-7a32-4ca2-b679-9b795f54e0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:22.791424 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b452d-7a32-4ca2-b679-9b795f54e0a3 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:22.792024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b452d-7a32-4ca2-b679-9b795f54e0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:22.792642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b452d-7a32-4ca2-b679-9b795f54e0a3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:22.793481 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b452d-7a32-4ca2-b679-9b795f54e0a3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:22.795432 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b452d-7a32-4ca2-b679-9b795f54e0a3 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:22.800389 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fbc482-d0be-47b5-91ce-f274d9ff0a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:22.801030 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fbc482-d0be-47b5-91ce-f274d9ff0a97 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:22.801306 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fbc482-d0be-47b5-91ce-f274d9ff0a97 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:22.801577 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fbc482-d0be-47b5-91ce-f274d9ff0a97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:22.807391 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-fd1b452d-7a32-4ca2-b679-9b795f54e0a3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 10:36:22.807391 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:22.807391 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:36:22.807391 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:36:22.807391 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:36:22.807391 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:36:22.807391 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:36:22.807391 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:22.807391 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 10:36:22.807391 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:22.807391 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:36:22.807391 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:22.807391 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:22.807391 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:22.807391 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:22.807391 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:22.807391 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:22.808419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:22.808419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:22.808419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:22.808419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:22.808419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:22.808419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:22.808419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:36:22.808419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:36:22.808419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:36:22.808419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:36:22.808419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:22.808419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:22.808419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:22.808419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:22.808419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:22.808419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:22.809277 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:22.809277 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:22.809277 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:22.809277 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:22.809277 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:22.809277 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:22.809277 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:22.809277 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:22.809277 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:22.809277 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:22.809277 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:36:22.809277 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:22.809277 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 10:36:22.809277 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:22.811143 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fbc482-d0be-47b5-91ce-f274d9ff0a97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:22.817455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fbc482-d0be-47b5-91ce-f274d9ff0a97 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:22.818017 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 427/832] 10.222.0.115 () {56 vars in 1119 bytes} [Tue Aug 2 10:36:22 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:36:22.819227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b452d-7a32-4ca2-b679-9b795f54e0a3 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:22.825387 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-b3fbc482-d0be-47b5-91ce-f274d9ff0a97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:22.825387 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:22.825387 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:22.825387 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:22.825387 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:22.825387 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:22.825387 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:22.825387 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:22.825387 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:22.825387 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:22.825387 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:22.825387 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:22.825387 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:36:22.825387 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:36:22.825387 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:36:22.826459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:36:22.826459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:22.826459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:22.826459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:22.826459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:22.826459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:22.826459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:22.826459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:22.826459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:22.826459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:22.826459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:22.826459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:22.826459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:22.826459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:22.826459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:22.826459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:22.826459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:22.827392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:36:22.827392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:36:22.827392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:36:22.827392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:36:22.827392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:36:22.827392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:36:22.827392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:22.827392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:22.827392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 406/833] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:36:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:36:22.836386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fbc482-d0be-47b5-91ce-f274d9ff0a97 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:22.939161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b452d-7a32-4ca2-b679-9b795f54e0a3 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:22.967982 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fbc482-d0be-47b5-91ce-f274d9ff0a97 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:22.981938 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd012ad-6bc7-471f-bf2a-31ba509e5c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:22.982514 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd012ad-6bc7-471f-bf2a-31ba509e5c8b None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:22.982798 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd012ad-6bc7-471f-bf2a-31ba509e5c8b None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:22.983129 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd012ad-6bc7-471f-bf2a-31ba509e5c8b None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:22.984164 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd012ad-6bc7-471f-bf2a-31ba509e5c8b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:22.984470 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd012ad-6bc7-471f-bf2a-31ba509e5c8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:22.986062 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd012ad-6bc7-471f-bf2a-31ba509e5c8b None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:22.991790 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 428/834] 10.222.0.115 () {56 vars in 1119 bytes} [Tue Aug 2 10:36:22 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:22.992994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f3422-00a1-4f5c-ba29-63159d0f8c28 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:22.993686 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f3422-00a1-4f5c-ba29-63159d0f8c28 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:22.993686 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6f3422-00a1-4f5c-ba29-63159d0f8c28 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:23.000503 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 429/835] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:23.000503 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f3422-00a1-4f5c-ba29-63159d0f8c28 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:23.024871 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:23.030056 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7900c306-ab6e-4607-9176-e8a7ca7184aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:23.030523 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:23.031806 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900c306-ab6e-4607-9176-e8a7ca7184aa None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:23.032218 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900c306-ab6e-4607-9176-e8a7ca7184aa None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:23.032723 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7900c306-ab6e-4607-9176-e8a7ca7184aa None admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:23.033275 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900c306-ab6e-4607-9176-e8a7ca7184aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:23.033586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900c306-ab6e-4607-9176-e8a7ca7184aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:23.035944 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7900c306-ab6e-4607-9176-e8a7ca7184aa None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:23.058527 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 407/836] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:36:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:23.058527 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980bf2b-f990-4233-881a-4fd18c68042c None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:23.058527 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980bf2b-f990-4233-881a-4fd18c68042c None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:23.058527 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980bf2b-f990-4233-881a-4fd18c68042c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:23.078740 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-3980bf2b-f990-4233-881a-4fd18c68042c None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:23.080334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-3980bf2b-f990-4233-881a-4fd18c68042c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:23.128298 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6f3422-00a1-4f5c-ba29-63159d0f8c28 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:23.155150 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 408/837] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:23 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:23.155150 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980bf2b-f990-4233-881a-4fd18c68042c None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:23.180982 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b2bad-3d34-49f5-a2f8-ae8a6412ddbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:23.182127 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b2bad-3d34-49f5-a2f8-ae8a6412ddbf None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:23.182950 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b2bad-3d34-49f5-a2f8-ae8a6412ddbf None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:23.183642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b2bad-3d34-49f5-a2f8-ae8a6412ddbf None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:23.184600 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b2bad-3d34-49f5-a2f8-ae8a6412ddbf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:23.185183 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b2bad-3d34-49f5-a2f8-ae8a6412ddbf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:23.187864 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27b2bad-3d34-49f5-a2f8-ae8a6412ddbf None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:23.193941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 430/838] 10.222.0.115 () {56 vars in 1119 bytes} [Tue Aug 2 10:36:22 2022] GET /identity/v3/services?type=volumev3 => generated 126 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:23.195569 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b2bad-3d34-49f5-a2f8-ae8a6412ddbf None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:23.267773 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980bf2b-f990-4233-881a-4fd18c68042c None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:23.297793 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:23.299415 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:23.314602 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0f5416-e3ac-4fc4-bbee-893ee4ae155d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:23.315568 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f5416-e3ac-4fc4-bbee-893ee4ae155d None admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:23.316126 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f5416-e3ac-4fc4-bbee-893ee4ae155d None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:23.316549 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0f5416-e3ac-4fc4-bbee-893ee4ae155d None admin] PATH_INFO: `/v3/projects/c1eae881e5714bf4bd223809e1a2d920/users/2b03263937cb4ca7807a24da44d37b0f/roles/0f1c61dab22c4d97ae784468058036c7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:23.325120 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b2bad-3d34-49f5-a2f8-ae8a6412ddbf None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:23.336386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0f5416-e3ac-4fc4-bbee-893ee4ae155d None admin] RBAC: Authorizing `identity:create_grant(project_id=c1eae881e5714bf4bd223809e1a2d920, user_id=2b03263937cb4ca7807a24da44d37b0f, role_id=0f1c61dab22c4d97ae784468058036c7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:23.340609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0f5416-e3ac-4fc4-bbee-893ee4ae155d None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:23.367270 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7918a22-e233-40ab-bf01-ded0dd31401e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:23.368055 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7918a22-e233-40ab-bf01-ded0dd31401e None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:23.368433 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7918a22-e233-40ab-bf01-ded0dd31401e None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:23.368769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7918a22-e233-40ab-bf01-ded0dd31401e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:23.369579 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7918a22-e233-40ab-bf01-ded0dd31401e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:23.371091 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7918a22-e233-40ab-bf01-ded0dd31401e None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:23.383309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-d7918a22-e233-40ab-bf01-ded0dd31401e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 10:36:23.383309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:23.383309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:36:23.383309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:36:23.383309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:36:23.383309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:36:23.383309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:36:23.383309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:23.383309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 10:36:23.383309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:23.383309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:36:23.383309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:23.383309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:23.383309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:23.383309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:23.383309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:23.384323 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:23.384323 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:23.384323 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:23.384323 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:23.384323 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:23.384323 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:23.384323 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:23.384323 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:36:23.384323 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:36:23.384323 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:36:23.384323 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:36:23.384323 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:23.384323 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:23.384323 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:23.384323 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:23.384323 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:23.385606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:23.385606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:23.385606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:23.385606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:23.385606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:23.385606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:23.385606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:23.385606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:23.385606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:23.385606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:23.385606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:23.385606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:36:23.385606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:23.385606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 10:36:23.385606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:23.385606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 431/839] 10.222.0.115 () {56 vars in 1137 bytes} [Tue Aug 2 10:36:23 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:23.390837 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7918a22-e233-40ab-bf01-ded0dd31401e None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:23.424456 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 409/840] 10.222.0.115 () {58 vars in 1444 bytes} [Tue Aug 2 10:36:23 2022] PUT /identity/v3/projects/c1eae881e5714bf4bd223809e1a2d920/users/2b03263937cb4ca7807a24da44d37b0f/roles/0f1c61dab22c4d97ae784468058036c7 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:36:23.538582 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7918a22-e233-40ab-bf01-ded0dd31401e None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:23.581412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdef923b-9622-4f75-97df-a5472db214e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:23.582144 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef923b-9622-4f75-97df-a5472db214e7 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:23.582489 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef923b-9622-4f75-97df-a5472db214e7 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:23.582828 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdef923b-9622-4f75-97df-a5472db214e7 None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:23.583652 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef923b-9622-4f75-97df-a5472db214e7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:23.584022 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef923b-9622-4f75-97df-a5472db214e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:23.585474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdef923b-9622-4f75-97df-a5472db214e7 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:23.592056 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 432/841] 10.222.0.115 () {56 vars in 1131 bytes} [Tue Aug 2 10:36:23 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:36:23.597702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0f5416-e3ac-4fc4-bbee-893ee4ae155d None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:23.742521 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0f5416-e3ac-4fc4-bbee-893ee4ae155d None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:23.784485 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7229fc5-aa02-437d-8c83-d7d96229ecf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:23.785262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7229fc5-aa02-437d-8c83-d7d96229ecf8 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:23.785649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7229fc5-aa02-437d-8c83-d7d96229ecf8 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:23.786021 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7229fc5-aa02-437d-8c83-d7d96229ecf8 None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:23.786922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7229fc5-aa02-437d-8c83-d7d96229ecf8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:23.787345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7229fc5-aa02-437d-8c83-d7d96229ecf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:23.789054 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7229fc5-aa02-437d-8c83-d7d96229ecf8 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:23.794888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 410/842] 10.222.0.115 () {56 vars in 1131 bytes} [Tue Aug 2 10:36:23 2022] GET /identity/v3/services?type=compute_legacy => generated 132 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:36:23.912243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec214c44-ae1c-464b-82d8-f33c240b0b7c None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:23.912525 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec214c44-ae1c-464b-82d8-f33c240b0b7c None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:23.912801 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec214c44-ae1c-464b-82d8-f33c240b0b7c None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:23.914497 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 433/843] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:23 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:23.920374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774eba3-d313-44d9-87b3-f7e5226c403f None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:23.920666 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774eba3-d313-44d9-87b3-f7e5226c403f None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:23.920942 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774eba3-d313-44d9-87b3-f7e5226c403f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:23.941722 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-a774eba3-d313-44d9-87b3-f7e5226c403f None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:23.943424 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-a774eba3-d313-44d9-87b3-f7e5226c403f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:23.985569 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:24.031493 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 411/844] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:23 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:36:24.057405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec214c44-ae1c-464b-82d8-f33c240b0b7c None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:24.071363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:24.170248 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec214c44-ae1c-464b-82d8-f33c240b0b7c None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:24.206597 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067c357-11c0-4169-96c9-09eb87d45113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:24.207278 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067c357-11c0-4169-96c9-09eb87d45113 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:24.207603 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067c357-11c0-4169-96c9-09eb87d45113 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:24.207874 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067c357-11c0-4169-96c9-09eb87d45113 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:24.220123 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067c357-11c0-4169-96c9-09eb87d45113 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:36:24.225071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-a067c357-11c0-4169-96c9-09eb87d45113 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:36:24.225071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:24.225071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:24.225071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:24.225071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:24.225071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:24.225071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:24.225071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:24.225071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:24.225071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:24.225071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:24.225071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:24.225071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:36:24.225071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:36:24.225071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:36:24.225071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:36:24.225997 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:36:24.225997 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:36:24.225997 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:36:24.225997 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:36:24.225997 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:24.225997 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:24.225997 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:24.225997 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:24.225997 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:24.225997 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:24.225997 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:24.225997 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:24.225997 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:24.225997 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:24.225997 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:24.225997 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:24.226845 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:24.226845 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:24.226845 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:24.226845 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:24.226845 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:36:24.226845 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:36:24.226845 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:36:24.226845 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:36:24.226845 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:36:24.226845 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:36:24.226845 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:36:24.226845 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:24.226845 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 434/845] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:36:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:24.232592 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a774eba3-d313-44d9-87b3-f7e5226c403f None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:24.347721 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a774eba3-d313-44d9-87b3-f7e5226c403f None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:24.385950 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c31d4-f54f-4a08-aabc-f41c26ccd781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:24.386611 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c31d4-f54f-4a08-aabc-f41c26ccd781 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:24.386899 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c31d4-f54f-4a08-aabc-f41c26ccd781 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:24.387214 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c31d4-f54f-4a08-aabc-f41c26ccd781 None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:24.388074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c31d4-f54f-4a08-aabc-f41c26ccd781 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:24.388890 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c31d4-f54f-4a08-aabc-f41c26ccd781 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:24.390396 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c31d4-f54f-4a08-aabc-f41c26ccd781 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:24.399479 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 412/846] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:36:24 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:24.404899 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067c357-11c0-4169-96c9-09eb87d45113 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:24.494469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:24.529092 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067c357-11c0-4169-96c9-09eb87d45113 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:24.564683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf3708-577f-472b-85cf-6850b64934e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:24.565316 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf3708-577f-472b-85cf-6850b64934e2 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:24.565594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf3708-577f-472b-85cf-6850b64934e2 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:24.565864 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf3708-577f-472b-85cf-6850b64934e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:24.577577 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bf3708-577f-472b-85cf-6850b64934e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:24.579015 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bf3708-577f-472b-85cf-6850b64934e2 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:24.594291 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-e6bf3708-577f-472b-85cf-6850b64934e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:24.594291 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:24.594291 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:24.594291 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:24.594291 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:24.594291 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:24.594291 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:24.594291 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:24.594291 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:24.594291 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:24.594291 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:24.594291 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:24.594291 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:24.594291 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:24.594291 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:24.595324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:24.595324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:24.595324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:24.595324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:24.595324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:24.595324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:24.595324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:24.595324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:24.595324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:24.595324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:24.595324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:24.595324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:24.595324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:24.595324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:24.595324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:24.595324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:24.595324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:24.596222 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:24.596222 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:24.596222 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:24.596222 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:24.596222 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 435/847] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:24.602193 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c31d4-f54f-4a08-aabc-f41c26ccd781 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:24.633286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:24.713512 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c31d4-f54f-4a08-aabc-f41c26ccd781 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:24.751936 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9793c7e-e429-4b73-a141-fe0d08711464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:24.752634 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9793c7e-e429-4b73-a141-fe0d08711464 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:24.752931 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9793c7e-e429-4b73-a141-fe0d08711464 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:24.753207 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9793c7e-e429-4b73-a141-fe0d08711464 None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:24.754006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9793c7e-e429-4b73-a141-fe0d08711464 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:24.754310 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9793c7e-e429-4b73-a141-fe0d08711464 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:24.755706 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9793c7e-e429-4b73-a141-fe0d08711464 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:24.770111 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 413/848] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:24 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:24.776590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf3708-577f-472b-85cf-6850b64934e2 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:24.919103 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf3708-577f-472b-85cf-6850b64934e2 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:24.952734 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:24.969104 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29725d3b-c464-44e6-b36a-b0b302d12079 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:24.970001 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29725d3b-c464-44e6-b36a-b0b302d12079 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:24.970401 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29725d3b-c464-44e6-b36a-b0b302d12079 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:24.970788 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29725d3b-c464-44e6-b36a-b0b302d12079 None admin] PATH_INFO: `/v3/users/swift` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:24.983165 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29725d3b-c464-44e6-b36a-b0b302d12079 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:24.985399 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29725d3b-c464-44e6-b36a-b0b302d12079 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:25.006293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-29725d3b-c464-44e6-b36a-b0b302d12079 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:36:25.006293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:25.006293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:25.006293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:25.006293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:25.006293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:25.006293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:25.006293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:25.006293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:25.006293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:25.006293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:25.006293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:25.006293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:36:25.006293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:36:25.006293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:36:25.006293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:36:25.007362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:25.007362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:25.007362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:36:25.007362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:25.007362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:36:25.007362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:25.007362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:25.007362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:25.007362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:25.007362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:25.007362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:25.007362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:25.007362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:25.007362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:25.007362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:25.007362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:25.007362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:25.007362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:25.008332 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:25.008332 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:25.008332 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:36:25.008332 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:36:25.008332 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:36:25.008332 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:36:25.008332 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:36:25.008332 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:36:25.008332 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 02 10:36:25.008332 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:25.008873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 436/849] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:36:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:25.015697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9793c7e-e429-4b73-a141-fe0d08711464 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:25.113332 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:25.126885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9793c7e-e429-4b73-a141-fe0d08711464 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:25.164419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e49af70-73bc-4b5a-aa56-c1d5fd988784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:25.165035 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49af70-73bc-4b5a-aa56-c1d5fd988784 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:25.165335 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49af70-73bc-4b5a-aa56-c1d5fd988784 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:25.165616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e49af70-73bc-4b5a-aa56-c1d5fd988784 None admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:25.166436 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e49af70-73bc-4b5a-aa56-c1d5fd988784 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:25.166745 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e49af70-73bc-4b5a-aa56-c1d5fd988784 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:25.168229 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e49af70-73bc-4b5a-aa56-c1d5fd988784 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:25.187160 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 414/850] 10.222.0.115 () {56 vars in 1140 bytes} [Tue Aug 2 10:36:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:25.193112 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29725d3b-c464-44e6-b36a-b0b302d12079 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:25.303973 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29725d3b-c464-44e6-b36a-b0b302d12079 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:25.340239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc93646-0029-4142-8d2c-cee01fbc3abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:25.340862 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc93646-0029-4142-8d2c-cee01fbc3abd None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:25.341170 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc93646-0029-4142-8d2c-cee01fbc3abd None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:25.341441 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc93646-0029-4142-8d2c-cee01fbc3abd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:25.411428 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:25.417637 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc93646-0029-4142-8d2c-cee01fbc3abd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:25.419109 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc93646-0029-4142-8d2c-cee01fbc3abd None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:25.484661 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894540b-8d64-492a-ba27-7db7844fd86a None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:25.484948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894540b-8d64-492a-ba27-7db7844fd86a None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:25.485230 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4894540b-8d64-492a-ba27-7db7844fd86a None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:25.486754 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 415/851] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:25.496447 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d2271-5a95-474b-bdc5-89008aa43f32 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:25.496728 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d2271-5a95-474b-bdc5-89008aa43f32 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:25.496999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704d2271-5a95-474b-bdc5-89008aa43f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:25.516497 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-704d2271-5a95-474b-bdc5-89008aa43f32 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:25.517698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-704d2271-5a95-474b-bdc5-89008aa43f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:25.529704 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 437/852] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:36:25 2022] GET /identity/v3/auth/tokens => generated 1163 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:36:25.536327 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc93646-0029-4142-8d2c-cee01fbc3abd None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:25.599140 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 416/853] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:25 2022] POST /identity/v3/auth/tokens => generated 1163 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:25.615324 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d2271-5a95-474b-bdc5-89008aa43f32 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:25.653034 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:25.676697 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc93646-0029-4142-8d2c-cee01fbc3abd None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:25.704723 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:25.725293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4ef8df-4332-4868-8876-562adc85d1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:25.725879 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ef8df-4332-4868-8876-562adc85d1b5 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:25.726163 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ef8df-4332-4868-8876-562adc85d1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:25.726447 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ef8df-4332-4868-8876-562adc85d1b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:25.737780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704d2271-5a95-474b-bdc5-89008aa43f32 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:25.744377 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ef8df-4332-4868-8876-562adc85d1b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:25.746134 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4ef8df-4332-4868-8876-562adc85d1b5 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:25.767227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-2a4ef8df-4332-4868-8876-562adc85d1b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:25.767227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:25.767227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:25.767227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:25.767227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:25.767227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:25.767227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:25.767227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:25.767227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:25.767227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:25.767227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:25.767227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:25.767227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:25.767227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:25.767227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:25.768459 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:25.768459 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:25.768459 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:25.768459 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:25.768459 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:25.768459 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:25.768459 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:25.768459 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:25.768459 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:25.768459 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:25.768459 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:25.768459 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:25.768459 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:25.768459 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:25.768459 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:25.768459 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:25.768459 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:25.769872 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:25.769872 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:25.769872 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:25.769872 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:25.769872 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 438/854] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:25.774657 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12dfe33-15bf-4731-9a5e-96bb542cd0e1 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:25.775221 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12dfe33-15bf-4731-9a5e-96bb542cd0e1 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:25.775579 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12dfe33-15bf-4731-9a5e-96bb542cd0e1 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:25.777882 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 439/855] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:25 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:36:25.778766 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12dfe33-15bf-4731-9a5e-96bb542cd0e1 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:25.789039 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74de2dbe-baed-42b6-bbee-8ea2f6f1cdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:25.789666 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de2dbe-baed-42b6-bbee-8ea2f6f1cdb9 None admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:25.790001 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de2dbe-baed-42b6-bbee-8ea2f6f1cdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:25.790275 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74de2dbe-baed-42b6-bbee-8ea2f6f1cdb9 None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:25.790969 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de2dbe-baed-42b6-bbee-8ea2f6f1cdb9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:25.792386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74de2dbe-baed-42b6-bbee-8ea2f6f1cdb9 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:25.883088 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 417/856] 10.222.0.115 () {60 vars in 1146 bytes} [Tue Aug 2 10:36:25 2022] POST /identity/v3/services => generated 252 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:36:25.883088 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f35977-11ad-4f06-bf27-503659092cbe None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:25.883088 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f35977-11ad-4f06-bf27-503659092cbe None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:25.883088 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f35977-11ad-4f06-bf27-503659092cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:25.902761 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12dfe33-15bf-4731-9a5e-96bb542cd0e1 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:25.907068 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-52f35977-11ad-4f06-bf27-503659092cbe None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:25.908397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-52f35977-11ad-4f06-bf27-503659092cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:25.945904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:25.955141 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416e84ea-7906-47e9-a7a5-283cc281c19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:25.959465 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e84ea-7906-47e9-a7a5-283cc281c19f None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:25.959465 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e84ea-7906-47e9-a7a5-283cc281c19f None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:25.959465 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416e84ea-7906-47e9-a7a5-283cc281c19f None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:25.959465 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416e84ea-7906-47e9-a7a5-283cc281c19f None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:25.959465 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416e84ea-7906-47e9-a7a5-283cc281c19f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:25.959465 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416e84ea-7906-47e9-a7a5-283cc281c19f None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:25.972736 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 440/857] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:25 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:25.973875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f5844-f472-471f-bbd6-d0d16f1259c8 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:25.974151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f5844-f472-471f-bbd6-d0d16f1259c8 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:25.974456 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f5844-f472-471f-bbd6-d0d16f1259c8 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:25.976062 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 441/858] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:25.980922 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f5844-f472-471f-bbd6-d0d16f1259c8 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:26.019597 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 418/859] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:25 2022] POST /identity/v3/auth/tokens => generated 1264 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:26.024408 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba92df2-8c16-4be7-9fef-3cdad2e170fa None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:26.025754 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba92df2-8c16-4be7-9fef-3cdad2e170fa None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:26.026190 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba92df2-8c16-4be7-9fef-3cdad2e170fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:26.049874 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-4ba92df2-8c16-4be7-9fef-3cdad2e170fa None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:26.051178 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-4ba92df2-8c16-4be7-9fef-3cdad2e170fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:26.124938 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f5844-f472-471f-bbd6-d0d16f1259c8 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:26.131101 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:26.151072 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 419/860] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:26 2022] POST /identity/v3/auth/tokens => generated 1264 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:26.151072 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba92df2-8c16-4be7-9fef-3cdad2e170fa None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:26.169042 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40dd84a-d58d-45f2-944a-a1f1ee7527c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:26.170491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dd84a-d58d-45f2-944a-a1f1ee7527c8 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:26.171196 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dd84a-d58d-45f2-944a-a1f1ee7527c8 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:26.171875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40dd84a-d58d-45f2-944a-a1f1ee7527c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:26.180461 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40dd84a-d58d-45f2-944a-a1f1ee7527c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:26.181914 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40dd84a-d58d-45f2-944a-a1f1ee7527c8 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:26.194769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-c40dd84a-d58d-45f2-944a-a1f1ee7527c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:26.194769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:26.194769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:26.194769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:26.194769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:26.194769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:26.194769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:26.194769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:26.194769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:26.194769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:26.194769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:26.194769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:26.194769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:36:26.194769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:36:26.194769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:36:26.196024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:36:26.196024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:26.196024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:26.196024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:26.196024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:26.196024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:26.196024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:26.196024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:26.196024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:26.196024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:26.196024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:26.196024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:26.196024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:26.196024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:26.196024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:26.196024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:26.196024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:26.197351 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:36:26.197351 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:36:26.197351 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:36:26.197351 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:36:26.197351 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:36:26.197351 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:36:26.197351 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:26.197351 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:26.199191 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 442/861] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:36:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:26.201484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40dd84a-d58d-45f2-944a-a1f1ee7527c8 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:26.264877 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:26.272587 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba92df2-8c16-4be7-9fef-3cdad2e170fa None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:26.317984 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974269b-1037-45fc-b416-22db2cce7782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:26.318639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974269b-1037-45fc-b416-22db2cce7782 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:26.318966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974269b-1037-45fc-b416-22db2cce7782 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:26.319287 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4974269b-1037-45fc-b416-22db2cce7782 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:26.328535 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40dd84a-d58d-45f2-944a-a1f1ee7527c8 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:26.330827 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4974269b-1037-45fc-b416-22db2cce7782 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:36:26.335891 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-4974269b-1037-45fc-b416-22db2cce7782 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:36:26.335891 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:26.335891 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:26.335891 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:26.335891 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:26.335891 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:26.335891 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:26.335891 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:26.335891 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:26.335891 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:26.335891 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:26.335891 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:26.335891 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:36:26.335891 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:36:26.335891 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:36:26.335891 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:36:26.336829 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:36:26.336829 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:36:26.336829 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:36:26.336829 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:36:26.336829 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:26.336829 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:26.336829 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:26.336829 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:26.336829 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:26.336829 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:26.336829 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:26.336829 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:26.336829 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:26.336829 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:26.336829 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:26.336829 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:26.337702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:26.337702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:26.337702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:26.337702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:26.337702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:36:26.337702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:36:26.337702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:36:26.337702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:36:26.337702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:36:26.337702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:36:26.337702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:36:26.337702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:26.337702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 420/862] 10.222.0.115 () {56 vars in 1107 bytes} [Tue Aug 2 10:36:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:26.339551 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974269b-1037-45fc-b416-22db2cce7782 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:26.374925 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a37c0-35bf-473d-a60e-3b08071af3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:26.375581 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a37c0-35bf-473d-a60e-3b08071af3a1 None admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:26.375861 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a37c0-35bf-473d-a60e-3b08071af3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:26.376132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a37c0-35bf-473d-a60e-3b08071af3a1 None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:26.376944 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a37c0-35bf-473d-a60e-3b08071af3a1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:26.378424 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a37c0-35bf-473d-a60e-3b08071af3a1 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:26.399551 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:26.453836 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4974269b-1037-45fc-b416-22db2cce7782 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:26.475095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 443/863] 10.222.0.115 () {60 vars in 1146 bytes} [Tue Aug 2 10:36:26 2022] POST /identity/v3/services => generated 270 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:36:26.475095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a37c0-35bf-473d-a60e-3b08071af3a1 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:26.515596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef623eb-0c06-413e-8bc2-e0db8f6b9437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:26.517387 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef623eb-0c06-413e-8bc2-e0db8f6b9437 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:26.517824 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef623eb-0c06-413e-8bc2-e0db8f6b9437 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:26.518244 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef623eb-0c06-413e-8bc2-e0db8f6b9437 None admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:26.519532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef623eb-0c06-413e-8bc2-e0db8f6b9437 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:26.520033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef623eb-0c06-413e-8bc2-e0db8f6b9437 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:26.522681 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef623eb-0c06-413e-8bc2-e0db8f6b9437 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:26.540978 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 421/864] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:36:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:26.548154 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef623eb-0c06-413e-8bc2-e0db8f6b9437 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:26.622185 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a37c0-35bf-473d-a60e-3b08071af3a1 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:26.636056 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:26.672837 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4e16c-11b7-4dc9-9f5e-735129a02d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:26.673762 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4e16c-11b7-4dc9-9f5e-735129a02d6c None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:26.674210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4e16c-11b7-4dc9-9f5e-735129a02d6c None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:26.674641 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4e16c-11b7-4dc9-9f5e-735129a02d6c None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:26.675642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4e16c-11b7-4dc9-9f5e-735129a02d6c None admin] RBAC: Adding query filter params (name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:26.676077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4e16c-11b7-4dc9-9f5e-735129a02d6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:26.677479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c4e16c-11b7-4dc9-9f5e-735129a02d6c None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:26.690129 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef623eb-0c06-413e-8bc2-e0db8f6b9437 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:26.691647 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 444/865] 10.222.0.115 () {56 vars in 1108 bytes} [Tue Aug 2 10:36:26 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:26.698171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4e16c-11b7-4dc9-9f5e-735129a02d6c None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:26.706912 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:26.746319 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d85354-b52e-43ec-98c7-432ef7275ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:26.746945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d85354-b52e-43ec-98c7-432ef7275ac1 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:26.747367 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d85354-b52e-43ec-98c7-432ef7275ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:26.747778 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d85354-b52e-43ec-98c7-432ef7275ac1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:26.748560 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d85354-b52e-43ec-98c7-432ef7275ac1 None admin] RBAC: Adding query filter params (role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac, scope.project.id=c1eae881e5714bf4bd223809e1a2d920, user.id=0daa49b5a35f4a468bb8d0939e7ec20c) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:26.748891 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d85354-b52e-43ec-98c7-432ef7275ac1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:26.750339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d85354-b52e-43ec-98c7-432ef7275ac1 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:26.764938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 422/866] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:36:26 2022] GET /identity/v3/role_assignments?user.id=0daa49b5a35f4a468bb8d0939e7ec20c&scope.project.id=c1eae881e5714bf4bd223809e1a2d920&role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 610 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:36:26.850723 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4e16c-11b7-4dc9-9f5e-735129a02d6c None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:26.895236 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91146a-3c57-41f7-a880-b17511b7661b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:26.896292 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91146a-3c57-41f7-a880-b17511b7661b None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:26.896777 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91146a-3c57-41f7-a880-b17511b7661b None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:26.897273 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91146a-3c57-41f7-a880-b17511b7661b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:26.910469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb91146a-3c57-41f7-a880-b17511b7661b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:26.912371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb91146a-3c57-41f7-a880-b17511b7661b None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:26.930155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-bb91146a-3c57-41f7-a880-b17511b7661b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:26.930155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:26.930155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:26.930155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:26.930155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:26.930155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:26.930155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:26.930155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:26.930155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:26.930155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:26.930155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:26.930155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:26.930155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:26.930155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:26.930155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:26.931387 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:26.931387 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:26.931387 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:26.931387 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:26.931387 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:26.931387 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:26.931387 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:26.931387 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:26.931387 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:26.931387 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:26.931387 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:26.931387 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:26.931387 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:26.931387 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:26.931387 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:26.931387 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:26.931387 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:26.932548 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:26.932548 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:26.932548 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:26.932548 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:26.934815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 445/867] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:26.939731 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d85354-b52e-43ec-98c7-432ef7275ac1 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:27.068467 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d85354-b52e-43ec-98c7-432ef7275ac1 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:27.106885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3ac33-0d71-4f65-983b-51aa0050c0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:27.107474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3ac33-0d71-4f65-983b-51aa0050c0cd None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:27.107964 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3ac33-0d71-4f65-983b-51aa0050c0cd None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:27.108247 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc3ac33-0d71-4f65-983b-51aa0050c0cd None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:27.108973 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3ac33-0d71-4f65-983b-51aa0050c0cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:27.109282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3ac33-0d71-4f65-983b-51aa0050c0cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:27.110669 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc3ac33-0d71-4f65-983b-51aa0050c0cd None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:27.127226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 423/868] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:26 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:27.132565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91146a-3c57-41f7-a880-b17511b7661b None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:27.170311 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:27.256448 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91146a-3c57-41f7-a880-b17511b7661b None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:27.300802 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9762fe9-c9ac-4d31-9674-06059f25bd40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:27.301834 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9762fe9-c9ac-4d31-9674-06059f25bd40 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:27.302343 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9762fe9-c9ac-4d31-9674-06059f25bd40 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:27.302812 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9762fe9-c9ac-4d31-9674-06059f25bd40 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:27.311648 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9762fe9-c9ac-4d31-9674-06059f25bd40 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:27.313856 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9762fe9-c9ac-4d31-9674-06059f25bd40 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:27.328804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-d9762fe9-c9ac-4d31-9674-06059f25bd40 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:36:27.328804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:27.328804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:27.328804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:27.328804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:27.328804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:27.328804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:27.328804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:27.328804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:27.328804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:27.328804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:27.328804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:27.328804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:36:27.328804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:36:27.328804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:36:27.328804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:36:27.330087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:27.330087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:27.330087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:36:27.330087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:27.330087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:36:27.330087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:27.330087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:27.330087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:27.330087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:27.330087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:27.330087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:27.330087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:27.330087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:27.330087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:27.330087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:27.330087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:27.330087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:27.330087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:27.331040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:27.331040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:27.331040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:36:27.331040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:36:27.331040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:36:27.331040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:36:27.331040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:36:27.331040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:36:27.331040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:36:27.331040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:27.333681 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 446/869] 10.222.0.115 () {56 vars in 1122 bytes} [Tue Aug 2 10:36:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:27.339536 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3ac33-0d71-4f65-983b-51aa0050c0cd None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:27.378303 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:27.468500 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc3ac33-0d71-4f65-983b-51aa0050c0cd None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:27.510789 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e494f6-c579-42d4-aaa4-99b6aef11e9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:27.511453 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e494f6-c579-42d4-aaa4-99b6aef11e9f None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:27.511732 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e494f6-c579-42d4-aaa4-99b6aef11e9f None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:27.512005 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e494f6-c579-42d4-aaa4-99b6aef11e9f None admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:27.512820 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e494f6-c579-42d4-aaa4-99b6aef11e9f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:27.513658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e494f6-c579-42d4-aaa4-99b6aef11e9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:27.515150 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e494f6-c579-42d4-aaa4-99b6aef11e9f None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:27.548657 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 424/870] 10.222.0.115 () {56 vars in 1154 bytes} [Tue Aug 2 10:36:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:27.557649 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9762fe9-c9ac-4d31-9674-06059f25bd40 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:27.679209 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:27.698940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9762fe9-c9ac-4d31-9674-06059f25bd40 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:27.741408 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7cb957-03ba-430c-98bd-da14e56ab52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:27.742085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7cb957-03ba-430c-98bd-da14e56ab52f None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:27.742373 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7cb957-03ba-430c-98bd-da14e56ab52f None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:27.742650 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7cb957-03ba-430c-98bd-da14e56ab52f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:27.849088 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7cb957-03ba-430c-98bd-da14e56ab52f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:27.851510 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7cb957-03ba-430c-98bd-da14e56ab52f None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:27.984476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:28.010782 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 447/871] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:36:27 2022] GET /identity/v3/auth/tokens => generated 1374 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:36:28.018086 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e494f6-c579-42d4-aaa4-99b6aef11e9f None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:28.140469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:28.147556 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e494f6-c579-42d4-aaa4-99b6aef11e9f None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:28.203531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ea678-3f4f-4f2d-bb8b-fe7b1ad0a0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:28.205198 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ea678-3f4f-4f2d-bb8b-fe7b1ad0a0ea None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:28.205685 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ea678-3f4f-4f2d-bb8b-fe7b1ad0a0ea None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:28.206143 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ea678-3f4f-4f2d-bb8b-fe7b1ad0a0ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:28.218141 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ea678-3f4f-4f2d-bb8b-fe7b1ad0a0ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:28.219883 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ea678-3f4f-4f2d-bb8b-fe7b1ad0a0ea None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:28.240104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-ca1ea678-3f4f-4f2d-bb8b-fe7b1ad0a0ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:28.240104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:28.240104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:28.240104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:28.240104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:28.240104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:28.240104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:28.240104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:28.240104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:28.240104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:28.240104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:28.240104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:28.240104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:28.240104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:28.240104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:28.241649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:28.241649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:28.241649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:28.241649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:28.241649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:28.241649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:28.241649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:28.241649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:28.241649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:28.241649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:28.241649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:28.241649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:28.241649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:28.241649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:28.241649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:28.241649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:28.241649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:28.243395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:28.243395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:28.243395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:28.243395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:28.246126 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 425/872] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:28.252226 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7cb957-03ba-430c-98bd-da14e56ab52f None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:28.314457 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba11acd-6fdd-4f60-8c69-3f064eeb42fb None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:28.314457 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba11acd-6fdd-4f60-8c69-3f064eeb42fb None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:28.314811 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba11acd-6fdd-4f60-8c69-3f064eeb42fb None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:28.319742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 426/873] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:36:28.321606 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fad770-e52d-4130-a608-f18b0eda8535 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:28.321965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fad770-e52d-4130-a608-f18b0eda8535 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:28.322414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fad770-e52d-4130-a608-f18b0eda8535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:28.347665 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-77fad770-e52d-4130-a608-f18b0eda8535 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:28.348954 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-77fad770-e52d-4130-a608-f18b0eda8535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:28.435424 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7cb957-03ba-430c-98bd-da14e56ab52f None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:28.442254 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 427/874] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:28 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:28.478169 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fad770-e52d-4130-a608-f18b0eda8535 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:28.496913 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320e221-f59e-4199-905d-22e9cd3974ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:28.497563 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320e221-f59e-4199-905d-22e9cd3974ed None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:28.497854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320e221-f59e-4199-905d-22e9cd3974ed None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:28.498136 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8320e221-f59e-4199-905d-22e9cd3974ed None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:28.499371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8320e221-f59e-4199-905d-22e9cd3974ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:28.499681 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8320e221-f59e-4199-905d-22e9cd3974ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:28.501085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8320e221-f59e-4199-905d-22e9cd3974ed None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:28.518245 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 448/875] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:28.523125 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320e221-f59e-4199-905d-22e9cd3974ed None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:28.592198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:28.600642 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:28.634323 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fad770-e52d-4130-a608-f18b0eda8535 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:28.668438 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8320e221-f59e-4199-905d-22e9cd3974ed None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:28.680690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c563fb-03d3-4c75-9b0a-98e666ee3559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:28.681344 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c563fb-03d3-4c75-9b0a-98e666ee3559 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:28.681616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c563fb-03d3-4c75-9b0a-98e666ee3559 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:28.681882 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c563fb-03d3-4c75-9b0a-98e666ee3559 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:28.682516 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c563fb-03d3-4c75-9b0a-98e666ee3559 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:28.683942 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c563fb-03d3-4c75-9b0a-98e666ee3559 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:28.695095 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-d0c563fb-03d3-4c75-9b0a-98e666ee3559 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 10:36:28.695095 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:28.695095 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:36:28.695095 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:36:28.695095 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:36:28.695095 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:36:28.695095 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:36:28.695095 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:28.695095 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 10:36:28.695095 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:28.695095 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:36:28.695095 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:28.695095 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:28.695095 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:28.695095 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:28.695095 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:28.695095 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:28.696286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:28.696286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:28.696286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:28.696286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:28.696286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:28.696286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:28.696286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:36:28.696286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:36:28.696286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:36:28.696286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:36:28.696286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:28.696286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:28.696286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:28.696286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:28.696286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:28.696286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:28.697172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:28.697172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:28.697172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:28.697172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:28.697172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:28.697172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:28.697172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:28.697172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:28.697172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:28.697172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:28.697172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:36:28.697172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:28.697172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 10:36:28.697172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:28.697172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 428/876] 10.222.0.115 () {56 vars in 1119 bytes} [Tue Aug 2 10:36:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:36:28.698160 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26bb2d8-b02a-4d33-a845-4122d752f88c None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:28.698160 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26bb2d8-b02a-4d33-a845-4122d752f88c None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:28.698160 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26bb2d8-b02a-4d33-a845-4122d752f88c None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:28.703128 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 429/877] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:28.703324 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26bb2d8-b02a-4d33-a845-4122d752f88c None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:28.725862 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45848a4c-dabf-4658-be53-9685d9eb1c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:28.726712 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45848a4c-dabf-4658-be53-9685d9eb1c2e None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:28.727140 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45848a4c-dabf-4658-be53-9685d9eb1c2e None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:28.727604 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45848a4c-dabf-4658-be53-9685d9eb1c2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:28.737919 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45848a4c-dabf-4658-be53-9685d9eb1c2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:28.740241 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45848a4c-dabf-4658-be53-9685d9eb1c2e None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:28.756329 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-45848a4c-dabf-4658-be53-9685d9eb1c2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:28.756329 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:28.756329 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:28.756329 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:28.756329 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:28.756329 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:28.756329 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:28.756329 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:28.756329 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:28.756329 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:28.756329 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:28.756329 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:28.756329 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:36:28.756329 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:36:28.756329 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:36:28.757470 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:36:28.757470 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:28.757470 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:28.757470 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:28.757470 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:28.757470 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:28.757470 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:28.757470 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:28.757470 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:28.757470 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:28.757470 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:28.757470 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:28.757470 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:28.757470 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:28.757470 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:28.757470 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:28.757470 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:28.758452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:36:28.758452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:36:28.758452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:36:28.758452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:36:28.758452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:36:28.758452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:36:28.758452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:28.758452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:28.758452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 449/878] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:36:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:36:28.758965 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc29d7f-dd3b-41df-886b-02f7c676bdf5 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:28.758965 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc29d7f-dd3b-41df-886b-02f7c676bdf5 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:28.759214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc29d7f-dd3b-41df-886b-02f7c676bdf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:28.780218 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-3dc29d7f-dd3b-41df-886b-02f7c676bdf5 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:28.781447 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-3dc29d7f-dd3b-41df-886b-02f7c676bdf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:28.834990 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26bb2d8-b02a-4d33-a845-4122d752f88c None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:28.875213 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 450/879] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:28 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:28.876364 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc29d7f-dd3b-41df-886b-02f7c676bdf5 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:28.880730 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1fdfe-2770-4922-81ee-3c84d08f0580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:28.881604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1fdfe-2770-4922-81ee-3c84d08f0580 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:28.882030 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1fdfe-2770-4922-81ee-3c84d08f0580 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:28.882502 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1fdfe-2770-4922-81ee-3c84d08f0580 None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:28.883362 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d1fdfe-2770-4922-81ee-3c84d08f0580 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:28.883980 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d1fdfe-2770-4922-81ee-3c84d08f0580 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:28.885739 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d1fdfe-2770-4922-81ee-3c84d08f0580 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:28.891669 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 430/880] 10.222.0.115 () {56 vars in 1119 bytes} [Tue Aug 2 10:36:28 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:36:28.892812 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1fdfe-2770-4922-81ee-3c84d08f0580 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:28.896791 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:28.930079 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:29.023902 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc29d7f-dd3b-41df-886b-02f7c676bdf5 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:29.024344 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1fdfe-2770-4922-81ee-3c84d08f0580 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:29.068898 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f920d-4644-4814-9a87-5c757ec9f9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:29.069727 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f920d-4644-4814-9a87-5c757ec9f9e6 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:29.070017 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f920d-4644-4814-9a87-5c757ec9f9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:29.070415 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9f920d-4644-4814-9a87-5c757ec9f9e6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:29.070724 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58a43b-336c-4b41-baf7-78469f7c96ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:29.071084 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f920d-4644-4814-9a87-5c757ec9f9e6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:29.071516 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58a43b-336c-4b41-baf7-78469f7c96ec None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:29.071797 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58a43b-336c-4b41-baf7-78469f7c96ec None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:29.072156 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb58a43b-336c-4b41-baf7-78469f7c96ec None admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:29.072472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9f920d-4644-4814-9a87-5c757ec9f9e6 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:29.073130 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb58a43b-336c-4b41-baf7-78469f7c96ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:29.073551 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb58a43b-336c-4b41-baf7-78469f7c96ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:29.075379 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb58a43b-336c-4b41-baf7-78469f7c96ec None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:29.082648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-eb9f920d-4644-4814-9a87-5c757ec9f9e6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 10:36:29.082648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:29.082648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:36:29.082648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:36:29.082648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:36:29.082648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:36:29.082648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:36:29.082648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:29.082648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 10:36:29.082648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:29.082648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:36:29.082648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:29.082648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:29.082648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:29.082648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:29.082648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:29.084580 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:29.084580 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:29.084580 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:29.084580 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:29.084580 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:29.084580 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:29.084580 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:29.084580 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:36:29.084580 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:36:29.084580 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:36:29.084580 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:36:29.084580 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:29.084580 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:29.084580 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:29.084580 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:29.084580 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:29.086102 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:29.086102 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:29.086102 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:29.086102 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:29.086102 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:29.086102 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:29.086102 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:29.086102 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:29.086102 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:29.086102 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:29.086102 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:29.086102 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:36:29.086102 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:29.086102 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 10:36:29.086102 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:29.086102 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 431/881] 10.222.0.115 () {56 vars in 1137 bytes} [Tue Aug 2 10:36:28 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:29.088304 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f920d-4644-4814-9a87-5c757ec9f9e6 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:29.097983 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 451/882] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:36:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:29.098683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58a43b-336c-4b41-baf7-78469f7c96ec None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:29.181378 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:29.219975 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9f920d-4644-4814-9a87-5c757ec9f9e6 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:29.238274 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:29.246204 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb58a43b-336c-4b41-baf7-78469f7c96ec None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:29.258077 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce9751-5bdf-4559-ae0b-386835b3dcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:29.258691 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce9751-5bdf-4559-ae0b-386835b3dcff None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:29.259157 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce9751-5bdf-4559-ae0b-386835b3dcff None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:29.259552 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ce9751-5bdf-4559-ae0b-386835b3dcff None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:29.260294 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce9751-5bdf-4559-ae0b-386835b3dcff None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:29.260599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce9751-5bdf-4559-ae0b-386835b3dcff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:29.261971 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ce9751-5bdf-4559-ae0b-386835b3dcff None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:29.275933 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 432/883] 10.222.0.115 () {56 vars in 1119 bytes} [Tue Aug 2 10:36:29 2022] GET /identity/v3/services?type=volumev3 => generated 364 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:29.275933 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce9751-5bdf-4559-ae0b-386835b3dcff None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:29.300185 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0e8ef1-0c0a-46ba-998e-45d5acf48d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:29.300185 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e8ef1-0c0a-46ba-998e-45d5acf48d3b None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:29.300185 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e8ef1-0c0a-46ba-998e-45d5acf48d3b None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:29.300185 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0e8ef1-0c0a-46ba-998e-45d5acf48d3b None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:29.302502 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0e8ef1-0c0a-46ba-998e-45d5acf48d3b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:29.302502 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0e8ef1-0c0a-46ba-998e-45d5acf48d3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:29.303603 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0e8ef1-0c0a-46ba-998e-45d5acf48d3b None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:29.308483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 452/884] 10.222.0.115 () {56 vars in 1131 bytes} [Tue Aug 2 10:36:29 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:29.309866 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307d0f7-bdf7-4ca6-8739-b6135a9c3664 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:29.311204 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307d0f7-bdf7-4ca6-8739-b6135a9c3664 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:29.311204 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7307d0f7-bdf7-4ca6-8739-b6135a9c3664 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:29.312749 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 453/885] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:29 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:29.313433 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307d0f7-bdf7-4ca6-8739-b6135a9c3664 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:29.403923 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ce9751-5bdf-4559-ae0b-386835b3dcff None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:29.436180 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:29.441654 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60895d-76d3-4ec5-b73b-edf1c2b468ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:29.442024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7307d0f7-bdf7-4ca6-8739-b6135a9c3664 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:29.442274 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60895d-76d3-4ec5-b73b-edf1c2b468ba None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:29.442560 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60895d-76d3-4ec5-b73b-edf1c2b468ba None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:29.442828 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60895d-76d3-4ec5-b73b-edf1c2b468ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:29.443863 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60895d-76d3-4ec5-b73b-edf1c2b468ba None admin] RBAC: Adding query filter params (role.id=0f1c61dab22c4d97ae784468058036c7, scope.project.id=c1eae881e5714bf4bd223809e1a2d920, user.id=2b03263937cb4ca7807a24da44d37b0f) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:29.444185 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60895d-76d3-4ec5-b73b-edf1c2b468ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:29.445870 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec60895d-76d3-4ec5-b73b-edf1c2b468ba None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:29.455263 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 433/886] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:36:29 2022] GET /identity/v3/role_assignments?user.id=2b03263937cb4ca7807a24da44d37b0f&scope.project.id=c1eae881e5714bf4bd223809e1a2d920&role.id=0f1c61dab22c4d97ae784468058036c7 => generated 610 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:29.455263 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60895d-76d3-4ec5-b73b-edf1c2b468ba None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:29.500282 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a864e672-2354-4952-88bb-bd6d43e617ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:29.503216 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864e672-2354-4952-88bb-bd6d43e617ea None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:29.503216 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864e672-2354-4952-88bb-bd6d43e617ea None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:29.503216 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a864e672-2354-4952-88bb-bd6d43e617ea None admin] PATH_INFO: `/v3/endpoints` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:29.503216 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864e672-2354-4952-88bb-bd6d43e617ea None admin] RBAC: Adding query filter params (interface=public, service_id=39ad6e55e751443b87f2737757edc350, region_id=RegionOne) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:29.503216 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864e672-2354-4952-88bb-bd6d43e617ea None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:29.504500 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a864e672-2354-4952-88bb-bd6d43e617ea None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:29.512751 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 454/887] 10.222.0.115 () {56 vars in 1256 bytes} [Tue Aug 2 10:36:29 2022] GET /identity/v3/endpoints?service_id=39ad6e55e751443b87f2737757edc350&interface=public®ion_id=RegionOne => generated 195 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:29.515218 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ec495-f5eb-4d2a-9f8a-3f23520cfd88 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:29.515698 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ec495-f5eb-4d2a-9f8a-3f23520cfd88 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:29.515986 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8ec495-f5eb-4d2a-9f8a-3f23520cfd88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:29.537350 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-6a8ec495-f5eb-4d2a-9f8a-3f23520cfd88 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:29.538522 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-6a8ec495-f5eb-4d2a-9f8a-3f23520cfd88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:29.563412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:29.613729 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60895d-76d3-4ec5-b73b-edf1c2b468ba None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:29.643672 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 455/888] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:29 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:29.644764 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8ec495-f5eb-4d2a-9f8a-3f23520cfd88 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:29.662379 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24b98c-28e3-4f86-ba7f-a0cfbf51e4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:29.663534 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24b98c-28e3-4f86-ba7f-a0cfbf51e4b6 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:29.663992 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24b98c-28e3-4f86-ba7f-a0cfbf51e4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:29.664570 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c24b98c-28e3-4f86-ba7f-a0cfbf51e4b6 None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:29.665256 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c24b98c-28e3-4f86-ba7f-a0cfbf51e4b6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:29.665558 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c24b98c-28e3-4f86-ba7f-a0cfbf51e4b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:29.667301 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c24b98c-28e3-4f86-ba7f-a0cfbf51e4b6 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:29.674987 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 434/889] 10.222.0.115 () {56 vars in 1131 bytes} [Tue Aug 2 10:36:29 2022] GET /identity/v3/services?type=compute_legacy => generated 388 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:29.676694 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24b98c-28e3-4f86-ba7f-a0cfbf51e4b6 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:29.754038 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:29.783931 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8ec495-f5eb-4d2a-9f8a-3f23520cfd88 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:29.809545 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c24b98c-28e3-4f86-ba7f-a0cfbf51e4b6 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:29.823812 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aaf6d5-b535-465b-a28c-4fd9c1b93753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:29.824442 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aaf6d5-b535-465b-a28c-4fd9c1b93753 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:29.824721 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aaf6d5-b535-465b-a28c-4fd9c1b93753 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:29.824997 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35aaf6d5-b535-465b-a28c-4fd9c1b93753 None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:29.826303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aaf6d5-b535-465b-a28c-4fd9c1b93753 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:29.827724 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35aaf6d5-b535-465b-a28c-4fd9c1b93753 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:29.843086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 456/890] 10.222.0.115 () {56 vars in 1092 bytes} [Tue Aug 2 10:36:29 2022] GET /identity/v3/services => generated 1274 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:36:29.843086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aaf6d5-b535-465b-a28c-4fd9c1b93753 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:29.857777 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:29.871900 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8226d-c380-4347-8440-c1d3dd1e770b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:29.877524 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8226d-c380-4347-8440-c1d3dd1e770b None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:29.877524 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8226d-c380-4347-8440-c1d3dd1e770b None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:29.877524 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8226d-c380-4347-8440-c1d3dd1e770b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:29.877524 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8226d-c380-4347-8440-c1d3dd1e770b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:29.878354 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8226d-c380-4347-8440-c1d3dd1e770b None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:29.895318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-c1b8226d-c380-4347-8440-c1d3dd1e770b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:36:29.895318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:29.895318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:36:29.895318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:36:29.895318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:36:29.895318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:36:29.895318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:36:29.895318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:29.895318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:36:29.895318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:29.895318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:36:29.895318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:29.895318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:29.895318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:29.895318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:29.895318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:29.895318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:29.896392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:29.896392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:29.896392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:29.896392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:29.896392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:29.896392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:29.896392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:36:29.896392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:36:29.896392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:36:29.896392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:36:29.896392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:29.896392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:29.896392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:29.896392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:29.896392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:29.896392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:29.897289 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:29.897289 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:29.897289 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:29.897289 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:29.897289 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:29.897289 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:29.897289 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:29.897289 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:29.897289 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:29.897289 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:29.897289 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:36:29.897289 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:29.897289 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:36:29.897289 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:29.899308 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 435/891] 10.222.0.115 () {56 vars in 1131 bytes} [Tue Aug 2 10:36:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:29.905577 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8226d-c380-4347-8440-c1d3dd1e770b None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:29.996795 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35aaf6d5-b535-465b-a28c-4fd9c1b93753 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:30.071476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0775c-be72-48b7-bff4-7f5291229bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:30.074175 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0775c-be72-48b7-bff4-7f5291229bde None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:30.074931 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0775c-be72-48b7-bff4-7f5291229bde None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:30.075708 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b0775c-be72-48b7-bff4-7f5291229bde None admin] PATH_INFO: `/v3/endpoints` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:30.075841 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8226d-c380-4347-8440-c1d3dd1e770b None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:30.077091 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b0775c-be72-48b7-bff4-7f5291229bde None admin] RBAC: Adding query filter params (interface=public, service_id=532297f0f4034902a35878144176a6df, region_id=RegionOne) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:30.077648 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b0775c-be72-48b7-bff4-7f5291229bde None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:30.080484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b0775c-be72-48b7-bff4-7f5291229bde None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:30.100265 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 457/892] 10.222.0.115 () {56 vars in 1256 bytes} [Tue Aug 2 10:36:29 2022] GET /identity/v3/endpoints?service_id=532297f0f4034902a35878144176a6df&interface=public®ion_id=RegionOne => generated 195 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:30.103026 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0775c-be72-48b7-bff4-7f5291229bde None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:30.120606 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:30.132857 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e005df8-a90f-4107-87c7-139c022c9ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:30.133605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e005df8-a90f-4107-87c7-139c022c9ca4 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:30.133864 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e005df8-a90f-4107-87c7-139c022c9ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:30.134730 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e005df8-a90f-4107-87c7-139c022c9ca4 None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:30.134840 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e005df8-a90f-4107-87c7-139c022c9ca4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:30.135165 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e005df8-a90f-4107-87c7-139c022c9ca4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:30.136573 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e005df8-a90f-4107-87c7-139c022c9ca4 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:30.142927 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 436/893] 10.222.0.115 () {56 vars in 1127 bytes} [Tue Aug 2 10:36:29 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:30.149003 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e005df8-a90f-4107-87c7-139c022c9ca4 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:30.227468 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b0775c-be72-48b7-bff4-7f5291229bde None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:30.234272 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:30.265032 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e005df8-a90f-4107-87c7-139c022c9ca4 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:30.276002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0575c7-8a51-4280-b1fd-fa70439cd474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:30.277440 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0575c7-8a51-4280-b1fd-fa70439cd474 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:30.278051 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0575c7-8a51-4280-b1fd-fa70439cd474 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:30.279363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0575c7-8a51-4280-b1fd-fa70439cd474 None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:30.280283 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0575c7-8a51-4280-b1fd-fa70439cd474 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:30.283230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0575c7-8a51-4280-b1fd-fa70439cd474 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:30.293150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 458/894] 10.222.0.115 () {56 vars in 1092 bytes} [Tue Aug 2 10:36:30 2022] GET /identity/v3/services => generated 1274 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:36:30.315087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b68ba7-c4a8-45b4-a472-b02c787d4b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:30.315087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b68ba7-c4a8-45b4-a472-b02c787d4b9c None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:30.315087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b68ba7-c4a8-45b4-a472-b02c787d4b9c None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:30.315087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b68ba7-c4a8-45b4-a472-b02c787d4b9c None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:30.315521 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b68ba7-c4a8-45b4-a472-b02c787d4b9c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:30.315521 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b68ba7-c4a8-45b4-a472-b02c787d4b9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:30.315983 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b68ba7-c4a8-45b4-a472-b02c787d4b9c None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:30.322081 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 437/895] 10.222.0.115 () {56 vars in 1127 bytes} [Tue Aug 2 10:36:30 2022] GET /identity/v3/services?type=object-store => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:32.017274 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b23f4-ec42-48d4-940f-aeef3d0ff059 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:32.018330 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b23f4-ec42-48d4-940f-aeef3d0ff059 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:32.018813 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b23f4-ec42-48d4-940f-aeef3d0ff059 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:32.020698 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 459/896] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:32 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:32.026388 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d5a59-31ee-474f-aa89-29a1032b2fd6 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:32.027347 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d5a59-31ee-474f-aa89-29a1032b2fd6 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:32.027835 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d5a59-31ee-474f-aa89-29a1032b2fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:32.048201 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-886d5a59-31ee-474f-aa89-29a1032b2fd6 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:32.049755 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-886d5a59-31ee-474f-aa89-29a1032b2fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:32.116878 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:32.149268 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 438/897] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:32 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:32.174831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b23f4-ec42-48d4-940f-aeef3d0ff059 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:32.291459 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b23f4-ec42-48d4-940f-aeef3d0ff059 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:32.334476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea01d80-86f6-4a7b-bdc4-423d8a46bef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:32.335528 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea01d80-86f6-4a7b-bdc4-423d8a46bef9 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:32.336034 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea01d80-86f6-4a7b-bdc4-423d8a46bef9 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:32.336500 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea01d80-86f6-4a7b-bdc4-423d8a46bef9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:32.349464 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea01d80-86f6-4a7b-bdc4-423d8a46bef9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:36:32.354881 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-7ea01d80-86f6-4a7b-bdc4-423d8a46bef9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:36:32.354881 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:32.354881 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:32.354881 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:32.354881 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:32.354881 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:32.354881 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:32.354881 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:32.354881 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:32.354881 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:32.354881 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:32.354881 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:32.354881 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:36:32.354881 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:36:32.354881 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:36:32.354881 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:36:32.355992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:36:32.355992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:36:32.355992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:36:32.355992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:36:32.355992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:32.355992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:32.355992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:32.355992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:32.355992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:32.355992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:32.355992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:32.355992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:32.355992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:32.355992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:32.355992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:32.355992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:32.357046 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:32.357046 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:32.357046 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:32.357046 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:32.357046 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:36:32.357046 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:36:32.357046 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:36:32.357046 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:36:32.357046 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:36:32.357046 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:36:32.357046 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:36:32.357046 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:32.363850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 460/898] 10.222.0.115 () {56 vars in 1125 bytes} [Tue Aug 2 10:36:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:32.368184 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d5a59-31ee-474f-aa89-29a1032b2fd6 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:32.453854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fff116-cf90-4139-89d3-42d4605dd8e1 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:32.454741 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fff116-cf90-4139-89d3-42d4605dd8e1 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:32.455249 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fff116-cf90-4139-89d3-42d4605dd8e1 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:32.463077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 461/899] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:32 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:32.467002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61631b35-c636-4572-a1ad-7e23da8ff2a5 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:32.467629 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61631b35-c636-4572-a1ad-7e23da8ff2a5 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:32.468121 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61631b35-c636-4572-a1ad-7e23da8ff2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:32.477741 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:32.495163 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-61631b35-c636-4572-a1ad-7e23da8ff2a5 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:32.496775 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-61631b35-c636-4572-a1ad-7e23da8ff2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:32.511236 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d5a59-31ee-474f-aa89-29a1032b2fd6 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:32.550952 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ab946-0e64-4242-b94b-0cbed18b069f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:32.551549 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ab946-0e64-4242-b94b-0cbed18b069f None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:32.551834 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ab946-0e64-4242-b94b-0cbed18b069f None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:32.552106 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ab946-0e64-4242-b94b-0cbed18b069f None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:32.552826 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ab946-0e64-4242-b94b-0cbed18b069f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:32.553130 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ab946-0e64-4242-b94b-0cbed18b069f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:32.554500 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ab946-0e64-4242-b94b-0cbed18b069f None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:32.571020 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 439/900] 10.222.0.115 () {56 vars in 1120 bytes} [Tue Aug 2 10:36:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:32.577019 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ab946-0e64-4242-b94b-0cbed18b069f None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:32.599075 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 462/901] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:32 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:32.619884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61631b35-c636-4572-a1ad-7e23da8ff2a5 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:32.655543 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:32.745727 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ab946-0e64-4242-b94b-0cbed18b069f None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:32.749058 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61631b35-c636-4572-a1ad-7e23da8ff2a5 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:32.786977 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42a6e13-6017-4b93-88bc-927fbcbde284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:32.787712 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a6e13-6017-4b93-88bc-927fbcbde284 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:32.787992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a6e13-6017-4b93-88bc-927fbcbde284 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:32.788265 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42a6e13-6017-4b93-88bc-927fbcbde284 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:32.789061 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42a6e13-6017-4b93-88bc-927fbcbde284 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:32.791253 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42a6e13-6017-4b93-88bc-927fbcbde284 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:32.791714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9eacc-4f4a-4635-9bd2-f5d2703b7485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:32.792357 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9eacc-4f4a-4635-9bd2-f5d2703b7485 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:32.792707 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9eacc-4f4a-4635-9bd2-f5d2703b7485 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:32.793180 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a9eacc-4f4a-4635-9bd2-f5d2703b7485 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:32.793870 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:32.815151 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9eacc-4f4a-4635-9bd2-f5d2703b7485 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:32.815151 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a9eacc-4f4a-4635-9bd2-f5d2703b7485 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:32.815497 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-d42a6e13-6017-4b93-88bc-927fbcbde284 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 10:36:32.815497 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:32.815497 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:36:32.815497 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:36:32.815497 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:36:32.815497 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:36:32.815497 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:36:32.815497 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:32.815497 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 10:36:32.815497 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:32.815497 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:36:32.815497 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:32.815497 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:32.815497 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:32.815497 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:32.815497 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:32.815497 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:32.816867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:32.816867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:32.816867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:32.816867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:32.816867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:32.816867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:32.816867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:36:32.816867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:36:32.816867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:36:32.816867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:36:32.816867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:32.816867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:32.816867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:32.816867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:32.816867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:32.816867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:32.818101 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:32.818101 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:32.818101 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:32.818101 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:32.818101 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:32.818101 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:32.818101 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:32.818101 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:32.818101 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:32.818101 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:32.818101 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:36:32.818101 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:32.818101 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 02 10:36:32.818101 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:32.818101 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 463/902] 10.222.0.115 () {56 vars in 1119 bytes} [Tue Aug 2 10:36:32 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:36:32.819232 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b31de1-5225-4f6e-a6ad-75584e685097 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:32.819232 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b31de1-5225-4f6e-a6ad-75584e685097 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:32.819232 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b31de1-5225-4f6e-a6ad-75584e685097 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:32.819232 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 464/903] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:32 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:32.819232 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474ca8d-1331-4c3e-97c8-5853e09122c8 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:32.819232 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474ca8d-1331-4c3e-97c8-5853e09122c8 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:32.819605 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474ca8d-1331-4c3e-97c8-5853e09122c8 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:32.819605 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 465/904] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:32 2022] GET /identity => generated 271 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:32.819605 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d10a5-46c1-467b-99ac-fb63cdc9b535 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:32.819605 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d10a5-46c1-467b-99ac-fb63cdc9b535 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:32.819605 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0d10a5-46c1-467b-99ac-fb63cdc9b535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:32.827277 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-34a9eacc-4f4a-4635-9bd2-f5d2703b7485 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:32.827277 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:32.827277 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:32.827277 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:32.827277 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:32.827277 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:32.827277 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:32.827277 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:32.827277 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:32.827277 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:32.827277 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:32.827277 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:32.827277 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:32.827277 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:32.827277 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:32.828474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:32.828474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:32.828474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:32.828474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:32.828474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:32.828474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:32.828474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:32.828474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:32.828474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:32.828474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:32.828474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:32.828474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:32.828474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:32.828474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:32.828474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:32.828474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:32.828474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:32.830123 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:32.830123 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:32.830123 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:32.830123 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:32.830123 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 440/905] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:32.830516 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9eacc-4f4a-4635-9bd2-f5d2703b7485 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:32.840385 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-ba0d10a5-46c1-467b-99ac-fb63cdc9b535 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:32.841888 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-ba0d10a5-46c1-467b-99ac-fb63cdc9b535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:32.922952 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 466/906] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:32 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:32.925278 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9e47b-f5bd-4fd3-81cc-08fc1b31a876 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:32.925278 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9e47b-f5bd-4fd3-81cc-08fc1b31a876 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:32.926978 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf9e47b-f5bd-4fd3-81cc-08fc1b31a876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:32.951615 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-7bf9e47b-f5bd-4fd3-81cc-08fc1b31a876 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:32.952814 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-7bf9e47b-f5bd-4fd3-81cc-08fc1b31a876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:32.960452 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:32.966877 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a9eacc-4f4a-4635-9bd2-f5d2703b7485 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:33.010559 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206f3bf-3c7c-49eb-a427-03ef53ff725f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:33.011304 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206f3bf-3c7c-49eb-a427-03ef53ff725f None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:33.011590 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206f3bf-3c7c-49eb-a427-03ef53ff725f None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:33.011865 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7206f3bf-3c7c-49eb-a427-03ef53ff725f None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:33.013129 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7206f3bf-3c7c-49eb-a427-03ef53ff725f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:33.013461 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7206f3bf-3c7c-49eb-a427-03ef53ff725f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:33.015098 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7206f3bf-3c7c-49eb-a427-03ef53ff725f None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:33.019901 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 441/907] 10.222.0.115 () {56 vars in 1119 bytes} [Tue Aug 2 10:36:32 2022] GET /identity/v3/services?name=volumev3 => generated 126 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:33.020898 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206f3bf-3c7c-49eb-a427-03ef53ff725f None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:33.030916 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 467/908] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:32 2022] POST /identity/v3/auth/tokens => generated 1374 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:33.031784 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9e47b-f5bd-4fd3-81cc-08fc1b31a876 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:33.133002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:33.146201 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7206f3bf-3c7c-49eb-a427-03ef53ff725f None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:33.180330 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf9e47b-f5bd-4fd3-81cc-08fc1b31a876 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:33.182797 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4dc3c3-2e29-47b6-a394-2b6478ddd075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:33.183483 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4dc3c3-2e29-47b6-a394-2b6478ddd075 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:33.183765 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4dc3c3-2e29-47b6-a394-2b6478ddd075 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:33.184042 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4dc3c3-2e29-47b6-a394-2b6478ddd075 None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:33.185219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4dc3c3-2e29-47b6-a394-2b6478ddd075 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:33.185518 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4dc3c3-2e29-47b6-a394-2b6478ddd075 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:33.186946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4dc3c3-2e29-47b6-a394-2b6478ddd075 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:33.201356 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 442/909] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:33 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:33.202277 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4dc3c3-2e29-47b6-a394-2b6478ddd075 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:33.225576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e471468-4843-48e7-9bc0-b505a4183189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:33.226202 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e471468-4843-48e7-9bc0-b505a4183189 None admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:33.226944 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e471468-4843-48e7-9bc0-b505a4183189 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:33.226944 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e471468-4843-48e7-9bc0-b505a4183189 None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:33.227479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e471468-4843-48e7-9bc0-b505a4183189 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:33.228857 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e471468-4843-48e7-9bc0-b505a4183189 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:33.229727 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:33.297272 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 468/910] 10.222.0.115 () {60 vars in 1146 bytes} [Tue Aug 2 10:36:33 2022] POST /identity/v3/services => generated 242 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:36:33.298241 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e471468-4843-48e7-9bc0-b505a4183189 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:33.323363 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4dc3c3-2e29-47b6-a394-2b6478ddd075 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:33.387828 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93436633-8244-42b3-a730-9930e586145e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:33.389333 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93436633-8244-42b3-a730-9930e586145e None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:33.389673 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93436633-8244-42b3-a730-9930e586145e None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:33.390131 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93436633-8244-42b3-a730-9930e586145e None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:33.391347 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93436633-8244-42b3-a730-9930e586145e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:33.391903 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93436633-8244-42b3-a730-9930e586145e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:33.394032 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93436633-8244-42b3-a730-9930e586145e None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:33.402355 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 443/911] 10.222.0.115 () {56 vars in 1119 bytes} [Tue Aug 2 10:36:33 2022] GET /identity/v3/services?type=volumev3 => generated 364 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:33.404300 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93436633-8244-42b3-a730-9930e586145e None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:33.464372 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e471468-4843-48e7-9bc0-b505a4183189 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:33.492341 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:33.521741 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:33.522604 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d02f8-12b1-43f4-bc08-cb963f02ef09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:33.523458 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d02f8-12b1-43f4-bc08-cb963f02ef09 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:33.523934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d02f8-12b1-43f4-bc08-cb963f02ef09 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:33.524377 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379d02f8-12b1-43f4-bc08-cb963f02ef09 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:33.525033 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379d02f8-12b1-43f4-bc08-cb963f02ef09 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:33.526413 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379d02f8-12b1-43f4-bc08-cb963f02ef09 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:33.537963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-379d02f8-12b1-43f4-bc08-cb963f02ef09 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 10:36:33.537963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:33.537963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:36:33.537963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:36:33.537963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:36:33.537963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:36:33.537963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:36:33.537963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:33.537963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 10:36:33.537963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:33.537963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:36:33.537963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:33.537963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:33.537963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:33.537963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:33.537963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:33.546821 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:33.546821 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:33.546821 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:33.546821 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:33.546821 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:33.546821 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:33.546821 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:33.546821 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:36:33.546821 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:36:33.546821 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:36:33.546821 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:36:33.546821 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:33.546821 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:33.546821 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:33.546821 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:33.546821 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:33.547793 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:33.547793 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:33.547793 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:33.547793 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:33.547793 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:33.547793 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:33.547793 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:33.547793 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:33.547793 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:33.547793 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:33.547793 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:33.547793 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:36:33.547793 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:33.547793 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 02 10:36:33.547793 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:33.547793 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 469/912] 10.222.0.115 () {56 vars in 1137 bytes} [Tue Aug 2 10:36:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:33.548655 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d02f8-12b1-43f4-bc08-cb963f02ef09 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:33.563164 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93436633-8244-42b3-a730-9930e586145e None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:33.603075 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477b46e-0cf3-4bdb-a48e-d5265a08672e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:33.604047 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477b46e-0cf3-4bdb-a48e-d5265a08672e None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:33.604839 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477b46e-0cf3-4bdb-a48e-d5265a08672e None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:33.604839 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6477b46e-0cf3-4bdb-a48e-d5265a08672e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:33.612645 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6477b46e-0cf3-4bdb-a48e-d5265a08672e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:33.614147 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6477b46e-0cf3-4bdb-a48e-d5265a08672e None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:33.626801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-6477b46e-0cf3-4bdb-a48e-d5265a08672e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:36:33.626801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:33.626801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:33.626801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:33.626801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:33.626801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:33.626801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:33.626801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:33.626801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:33.626801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:33.626801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:33.626801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:33.626801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:36:33.626801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:36:33.626801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:36:33.626801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:36:33.627945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:33.627945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:33.627945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:36:33.627945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:33.627945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:36:33.627945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:33.627945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:33.627945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:33.627945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:33.627945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:33.627945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:33.627945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:33.627945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:33.627945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:33.627945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:33.627945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:33.627945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:33.627945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:33.628903 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:33.628903 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:33.628903 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:36:33.628903 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:36:33.628903 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:36:33.628903 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:36:33.628903 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:36:33.628903 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:36:33.628903 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:36:33.628903 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:33.628903 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 444/913] 10.222.0.115 () {56 vars in 1122 bytes} [Tue Aug 2 10:36:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:33.629502 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477b46e-0cf3-4bdb-a48e-d5265a08672e None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:33.685523 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379d02f8-12b1-43f4-bc08-cb963f02ef09 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:33.740535 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51940759-a5df-49b7-b4c9-e946c92ca6fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:33.742897 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51940759-a5df-49b7-b4c9-e946c92ca6fd None admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:33.743329 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51940759-a5df-49b7-b4c9-e946c92ca6fd None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:33.743509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51940759-a5df-49b7-b4c9-e946c92ca6fd None admin] PATH_INFO: `/v3/endpoints` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:33.744209 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51940759-a5df-49b7-b4c9-e946c92ca6fd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:33.745594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51940759-a5df-49b7-b4c9-e946c92ca6fd None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:33.765382 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6477b46e-0cf3-4bdb-a48e-d5265a08672e None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:33.806652 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27ad90-fe28-4861-b02b-b1148414be8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:33.807228 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27ad90-fe28-4861-b02b-b1148414be8d None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:33.807693 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27ad90-fe28-4861-b02b-b1148414be8d None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:33.808032 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f27ad90-fe28-4861-b02b-b1148414be8d None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:33.808444 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f27ad90-fe28-4861-b02b-b1148414be8d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:33.808754 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f27ad90-fe28-4861-b02b-b1148414be8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:33.810838 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f27ad90-fe28-4861-b02b-b1148414be8d None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:33.815570 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 445/914] 10.222.0.115 () {56 vars in 1131 bytes} [Tue Aug 2 10:36:33 2022] GET /identity/v3/services?name=compute_legacy => generated 132 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:33.816369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27ad90-fe28-4861-b02b-b1148414be8d None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:33.821980 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:33.849069 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 470/915] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:36:33 2022] POST /identity/v3/endpoints => generated 348 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:36:33.849966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51940759-a5df-49b7-b4c9-e946c92ca6fd None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:33.951133 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:33.980172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f27ad90-fe28-4861-b02b-b1148414be8d None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:34.058208 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c95b33-3012-40f6-abc7-000ee1ab4763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:34.058208 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c95b33-3012-40f6-abc7-000ee1ab4763 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:34.058488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51940759-a5df-49b7-b4c9-e946c92ca6fd None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:34.058924 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c95b33-3012-40f6-abc7-000ee1ab4763 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:34.058924 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c95b33-3012-40f6-abc7-000ee1ab4763 None admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:34.058924 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c95b33-3012-40f6-abc7-000ee1ab4763 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:34.058924 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c95b33-3012-40f6-abc7-000ee1ab4763 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:34.067899 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c95b33-3012-40f6-abc7-000ee1ab4763 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:34.088443 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 446/916] 10.222.0.115 () {56 vars in 1154 bytes} [Tue Aug 2 10:36:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:34.094802 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c95b33-3012-40f6-abc7-000ee1ab4763 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:34.124240 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda782f7-9292-4db0-8820-3aad3eb76ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:34.124884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda782f7-9292-4db0-8820-3aad3eb76ad8 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:34.125168 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda782f7-9292-4db0-8820-3aad3eb76ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:34.125690 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda782f7-9292-4db0-8820-3aad3eb76ad8 None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:34.127373 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda782f7-9292-4db0-8820-3aad3eb76ad8 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:34.127909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda782f7-9292-4db0-8820-3aad3eb76ad8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:34.130671 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda782f7-9292-4db0-8820-3aad3eb76ad8 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:34.141966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 471/917] 10.222.0.115 () {56 vars in 1131 bytes} [Tue Aug 2 10:36:33 2022] GET /identity/v3/services?type=compute_legacy => generated 388 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:34.148175 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda782f7-9292-4db0-8820-3aad3eb76ad8 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:34.200583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:34.267224 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c95b33-3012-40f6-abc7-000ee1ab4763 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:34.321792 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:34.322523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d7a01-2e0b-4b61-8e36-7f94a2972548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:34.323638 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d7a01-2e0b-4b61-8e36-7f94a2972548 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:34.324199 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d7a01-2e0b-4b61-8e36-7f94a2972548 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:34.324659 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d7a01-2e0b-4b61-8e36-7f94a2972548 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:34.327576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda782f7-9292-4db0-8820-3aad3eb76ad8 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:34.380566 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9c336-6bd7-4b5c-a47c-f542088f1c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:34.381689 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9c336-6bd7-4b5c-a47c-f542088f1c90 None admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:34.382255 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9c336-6bd7-4b5c-a47c-f542088f1c90 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:34.382942 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e9c336-6bd7-4b5c-a47c-f542088f1c90 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:34.384596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e9c336-6bd7-4b5c-a47c-f542088f1c90 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:34.387036 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e9c336-6bd7-4b5c-a47c-f542088f1c90 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:34.424723 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d7a01-2e0b-4b61-8e36-7f94a2972548 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:34.426407 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d7a01-2e0b-4b61-8e36-7f94a2972548 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:34.491809 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 472/918] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:36:34 2022] POST /identity/v3/endpoints => generated 349 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:36:34.545426 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:34.629768 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 447/919] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:36:34 2022] GET /identity/v3/auth/tokens => generated 1476 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 10:36:34.636065 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9c336-6bd7-4b5c-a47c-f542088f1c90 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:34.774373 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e9c336-6bd7-4b5c-a47c-f542088f1c90 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:34.824619 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6d9c8-f161-4eb8-8176-eee26252b5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:34.825635 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6d9c8-f161-4eb8-8176-eee26252b5e8 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:34.826149 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6d9c8-f161-4eb8-8176-eee26252b5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:34.826660 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6d9c8-f161-4eb8-8176-eee26252b5e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:34.845583 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a6d9c8-f161-4eb8-8176-eee26252b5e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:34.848357 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a6d9c8-f161-4eb8-8176-eee26252b5e8 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:34.875211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-c2a6d9c8-f161-4eb8-8176-eee26252b5e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:34.875211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:34.875211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:34.875211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:34.875211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:34.875211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:34.875211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:34.875211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:34.875211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:34.875211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:34.875211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:34.875211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:34.875211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:34.875211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:34.875211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:34.876156 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:34.876156 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:34.876156 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:34.876156 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:34.876156 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:34.876156 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:34.876156 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:34.876156 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:34.876156 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:34.876156 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:34.876156 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:34.876156 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:34.876156 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:34.876156 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:34.876156 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:34.876156 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:34.876156 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:34.877037 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:34.877037 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:34.877037 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:34.877037 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:34.877441 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 473/920] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:34.883431 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d7a01-2e0b-4b61-8e36-7f94a2972548 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:34.992963 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d7a01-2e0b-4b61-8e36-7f94a2972548 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:35.032065 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d29733-00b7-4a23-b704-17a447166448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:35.033042 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d29733-00b7-4a23-b704-17a447166448 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:35.033544 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d29733-00b7-4a23-b704-17a447166448 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:35.034017 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d29733-00b7-4a23-b704-17a447166448 None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:35.034926 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d29733-00b7-4a23-b704-17a447166448 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:35.035451 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d29733-00b7-4a23-b704-17a447166448 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:35.037096 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d29733-00b7-4a23-b704-17a447166448 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:35.050640 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 448/921] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:34 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:35.056518 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6d9c8-f161-4eb8-8176-eee26252b5e8 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:35.059817 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:35.185405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6d9c8-f161-4eb8-8176-eee26252b5e8 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:35.223719 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48282f57-2d57-4f50-af80-6bcff0f385ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:35.224567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48282f57-2d57-4f50-af80-6bcff0f385ad None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:35.224912 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48282f57-2d57-4f50-af80-6bcff0f385ad None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:35.225254 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48282f57-2d57-4f50-af80-6bcff0f385ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:35.233732 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48282f57-2d57-4f50-af80-6bcff0f385ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:35.235702 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48282f57-2d57-4f50-af80-6bcff0f385ad None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:35.248469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-48282f57-2d57-4f50-af80-6bcff0f385ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:35.248469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:35.248469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:35.248469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:35.248469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:35.248469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:35.248469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:35.248469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:35.248469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:35.248469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:35.248469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:35.248469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:35.248469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:36:35.248469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:36:35.248469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:36:35.249599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:36:35.249599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:35.249599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:35.249599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:35.249599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:35.249599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:35.249599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:35.249599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:35.249599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:35.249599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:35.249599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:35.249599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:35.249599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:35.249599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:35.249599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:35.249599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:35.249599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:35.250723 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:36:35.250723 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:36:35.250723 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:36:35.250723 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:36:35.250723 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:36:35.250723 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:36:35.250723 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:35.250723 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:35.251731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 474/922] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:36:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:35.257664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d29733-00b7-4a23-b704-17a447166448 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:35.280907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:35.371204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d29733-00b7-4a23-b704-17a447166448 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:35.414980 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2c9f71-fdf0-4a10-a485-ef767f08e55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:35.416253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c9f71-fdf0-4a10-a485-ef767f08e55b None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:35.416863 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c9f71-fdf0-4a10-a485-ef767f08e55b None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:35.417472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2c9f71-fdf0-4a10-a485-ef767f08e55b None admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:35.418852 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c9f71-fdf0-4a10-a485-ef767f08e55b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:35.420206 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c9f71-fdf0-4a10-a485-ef767f08e55b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:35.422815 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2c9f71-fdf0-4a10-a485-ef767f08e55b None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:35.441374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 449/923] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:36:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:35.447837 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48282f57-2d57-4f50-af80-6bcff0f385ad None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:35.535845 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:35.594065 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48282f57-2d57-4f50-af80-6bcff0f385ad None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:35.631838 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1cbb96-7c96-496d-bb22-efd935c3f02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:35.632456 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1cbb96-7c96-496d-bb22-efd935c3f02f None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:35.632740 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1cbb96-7c96-496d-bb22-efd935c3f02f None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:35.633024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1cbb96-7c96-496d-bb22-efd935c3f02f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:35.633798 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1cbb96-7c96-496d-bb22-efd935c3f02f None admin] RBAC: Adding query filter params (role.id=7fee85e1d15545a1926ac22137e06585, scope.project.id=c1eae881e5714bf4bd223809e1a2d920, user.id=2b03263937cb4ca7807a24da44d37b0f) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:35.634099 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1cbb96-7c96-496d-bb22-efd935c3f02f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:35.635646 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1cbb96-7c96-496d-bb22-efd935c3f02f None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:35.642282 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 475/924] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:36:35 2022] GET /identity/v3/role_assignments?user.id=2b03263937cb4ca7807a24da44d37b0f&scope.project.id=c1eae881e5714bf4bd223809e1a2d920&role.id=7fee85e1d15545a1926ac22137e06585 => generated 260 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:35.745029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1bcf8-e729-46d4-81ef-2fc148c0553f None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:35.745263 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1bcf8-e729-46d4-81ef-2fc148c0553f None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:35.745612 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1bcf8-e729-46d4-81ef-2fc148c0553f None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:35.747276 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 450/925] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:35 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:35.754860 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8a259-bc1b-499d-902f-f5966fd673d9 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:35.755541 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8a259-bc1b-499d-902f-f5966fd673d9 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:35.756156 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8a259-bc1b-499d-902f-f5966fd673d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:35.791493 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-e4e8a259-bc1b-499d-902f-f5966fd673d9 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:35.792736 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-e4e8a259-bc1b-499d-902f-f5966fd673d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:35.883233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 476/926] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:35 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:35.908885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1bcf8-e729-46d4-81ef-2fc148c0553f None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:36.039161 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:36.082316 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1bcf8-e729-46d4-81ef-2fc148c0553f None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:36.144818 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf78111-771b-498a-90e7-41d378bdfe5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:36.145894 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf78111-771b-498a-90e7-41d378bdfe5b None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:36.146379 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf78111-771b-498a-90e7-41d378bdfe5b None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:36.146858 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf78111-771b-498a-90e7-41d378bdfe5b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:36.148009 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf78111-771b-498a-90e7-41d378bdfe5b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:36.150481 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf78111-771b-498a-90e7-41d378bdfe5b None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:36.167686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-edf78111-771b-498a-90e7-41d378bdfe5b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:36:36.167686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:36.167686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:36:36.167686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:36:36.167686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:36:36.167686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:36:36.167686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:36:36.167686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:36.167686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:36:36.167686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:36.167686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:36:36.167686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:36.167686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:36.167686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:36.167686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:36.167686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:36.167686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:36.169672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:36.169672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:36.169672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:36.169672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:36.169672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:36.169672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:36.169672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:36:36.169672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:36:36.169672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:36:36.169672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:36:36.169672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:36.169672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:36.169672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:36.169672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:36.169672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:36.169672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:36.171333 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:36.171333 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:36.171333 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:36.171333 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:36.171333 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:36.171333 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:36.171333 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:36.171333 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:36.171333 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:36.171333 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:36.171333 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:36:36.171333 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:36.171333 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:36:36.171333 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:36.171333 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 451/927] 10.222.0.115 () {56 vars in 1131 bytes} [Tue Aug 2 10:36:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:36.179862 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8a259-bc1b-499d-902f-f5966fd673d9 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:36.272786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b097f-7009-4eff-b181-3f23e0aa8116 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:36.273334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b097f-7009-4eff-b181-3f23e0aa8116 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:36.273334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348b097f-7009-4eff-b181-3f23e0aa8116 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:36.275188 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 452/928] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:36:36.287301 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5602234-c019-4d57-993e-a109b5aa6adf None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:36.287301 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5602234-c019-4d57-993e-a109b5aa6adf None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:36.287301 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5602234-c019-4d57-993e-a109b5aa6adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:36.291591 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:36.306081 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-d5602234-c019-4d57-993e-a109b5aa6adf None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:36.307815 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-d5602234-c019-4d57-993e-a109b5aa6adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:36.353877 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8a259-bc1b-499d-902f-f5966fd673d9 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:36.403542 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 453/929] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:36 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:36:36.405021 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d34bc50-47ed-41d4-9d47-fb290c0cdf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:36.405818 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34bc50-47ed-41d4-9d47-fb290c0cdf13 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:36.406153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34bc50-47ed-41d4-9d47-fb290c0cdf13 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:36.406428 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d34bc50-47ed-41d4-9d47-fb290c0cdf13 None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:36.407161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d34bc50-47ed-41d4-9d47-fb290c0cdf13 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:36.407469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d34bc50-47ed-41d4-9d47-fb290c0cdf13 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:36.408846 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d34bc50-47ed-41d4-9d47-fb290c0cdf13 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:36.415016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 477/930] 10.222.0.115 () {56 vars in 1127 bytes} [Tue Aug 2 10:36:36 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:36.424263 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5602234-c019-4d57-993e-a109b5aa6adf None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:36.439632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d34bc50-47ed-41d4-9d47-fb290c0cdf13 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:36.511142 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:36.546087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5602234-c019-4d57-993e-a109b5aa6adf None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:36.580340 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d34bc50-47ed-41d4-9d47-fb290c0cdf13 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:36.584597 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270b70b-caa5-46ed-b213-8d3a73c79044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:36.586260 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270b70b-caa5-46ed-b213-8d3a73c79044 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:36.586260 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270b70b-caa5-46ed-b213-8d3a73c79044 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:36.586260 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270b70b-caa5-46ed-b213-8d3a73c79044 None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:36.586589 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1270b70b-caa5-46ed-b213-8d3a73c79044 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:36.586830 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1270b70b-caa5-46ed-b213-8d3a73c79044 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:36.588381 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1270b70b-caa5-46ed-b213-8d3a73c79044 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:36.598599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 454/931] 10.222.0.115 () {56 vars in 1127 bytes} [Tue Aug 2 10:36:36 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:36.604777 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270b70b-caa5-46ed-b213-8d3a73c79044 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:36.627466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:36.657436 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78363b-1608-4dc1-aedb-24444fe4cddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:36.658124 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78363b-1608-4dc1-aedb-24444fe4cddd None admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:36.658457 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78363b-1608-4dc1-aedb-24444fe4cddd None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:36.658742 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c78363b-1608-4dc1-aedb-24444fe4cddd None admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:36.659484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c78363b-1608-4dc1-aedb-24444fe4cddd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:36.661517 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c78363b-1608-4dc1-aedb-24444fe4cddd None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:36.664276 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: WARNING keystone.server.flask.common [None req-9c78363b-1608-4dc1-aedb-24444fe4cddd None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 02 10:36:36.728797 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.sql.core [None req-9c78363b-1608-4dc1-aedb-24444fe4cddd None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 02 10:36:36.728797 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 02 10:36:36.728797 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [parameters: {'id': '2581cb541a2441e2a318fc2657d7fb25', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 02 10:36:36.728797 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=92110) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 02 10:36:36.736951 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-9c78363b-1608-4dc1-aedb-24444fe4cddd None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2581cb541a2441e2a318fc2657d7fb25.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2581cb541a2441e2a318fc2657d7fb25. Aug 02 10:36:36.736951 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:36.736951 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 10:36:36.736951 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 10:36:36.736951 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 10:36:36.736951 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 10:36:36.736951 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 10:36:36.736951 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 10:36:36.736951 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 10:36:36.736951 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application conn.query(q) Aug 02 10:36:36.736951 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 10:36:36.736951 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 10:36:36.736951 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 10:36:36.746924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application result.read() Aug 02 10:36:36.746924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 10:36:36.746924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 10:36:36.746924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 10:36:36.746924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 10:36:36.746924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 10:36:36.746924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 10:36:36.746924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 10:36:36.746924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 10:36:36.746924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 02 10:36:36.746924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:36.746924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 02 10:36:36.746924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:36.746924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:36.746924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 02 10:36:36.746924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 02 10:36:36.746924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 02 10:36:36.746924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 02 10:36:36.750683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 10:36:36.750683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application next(self.gen) Aug 02 10:36:36.750683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 02 10:36:36.750683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application yield resource Aug 02 10:36:36.750683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 10:36:36.750683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application next(self.gen) Aug 02 10:36:36.750683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 02 10:36:36.750683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self.session.rollback() Aug 02 10:36:36.750683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 02 10:36:36.750683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self.force_reraise() Aug 02 10:36:36.750683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 02 10:36:36.750683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise self.value Aug 02 10:36:36.750683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 02 10:36:36.750683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 02 10:36:36.750683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 02 10:36:36.750683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application session.commit() Aug 02 10:36:36.750683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 02 10:36:36.753295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 02 10:36:36.753295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 02 10:36:36.753295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._prepare_impl() Aug 02 10:36:36.753295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 02 10:36:36.753295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self.session.flush() Aug 02 10:36:36.753295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 02 10:36:36.753295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._flush(objects) Aug 02 10:36:36.753295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 02 10:36:36.753295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 02 10:36:36.753295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 02 10:36:36.753295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application compat.raise_( Aug 02 10:36:36.753295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 10:36:36.753295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception Aug 02 10:36:36.753295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 02 10:36:36.753295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application flush_context.execute() Aug 02 10:36:36.753295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 02 10:36:36.753295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rec.execute(self) Aug 02 10:36:36.756395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 02 10:36:36.756395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 02 10:36:36.756395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 02 10:36:36.756395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 02 10:36:36.756395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 02 10:36:36.756395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 02 10:36:36.756395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 02 10:36:36.756395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 02 10:36:36.756395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 02 10:36:36.756395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 02 10:36:36.756395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 02 10:36:36.756395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 02 10:36:36.756395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 02 10:36:36.756395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 02 10:36:36.756395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 02 10:36:36.758834 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 02 10:36:36.758834 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 10:36:36.758834 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception Aug 02 10:36:36.758834 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 10:36:36.758834 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 10:36:36.758834 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 10:36:36.758834 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 10:36:36.758834 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 10:36:36.758834 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 10:36:36.758834 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 10:36:36.758834 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application conn.query(q) Aug 02 10:36:36.758834 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 10:36:36.758834 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 10:36:36.758834 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 10:36:36.758834 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application result.read() Aug 02 10:36:36.758834 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 10:36:36.758834 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 10:36:36.761053 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 10:36:36.761053 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 10:36:36.761053 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 10:36:36.761053 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 10:36:36.761053 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 10:36:36.761053 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 10:36:36.761053 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 02 10:36:36.761053 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 02 10:36:36.761053 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [parameters: {'id': '2581cb541a2441e2a318fc2657d7fb25', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 02 10:36:36.761053 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 02 10:36:36.761053 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:36.761053 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:36:36.761053 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:36.761053 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:36.761053 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 02 10:36:36.763598 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 02 10:36:36.763598 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 02 10:36:36.763598 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 02 10:36:36.763598 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 02 10:36:36.763598 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:36.763598 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:36:36.763598 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:36.763598 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:36.763598 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:36.763598 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:36.763598 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:36.763598 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:36.763598 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:36.763598 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:36.763598 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:36.763598 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:36.763598 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:36.766317 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:36.766317 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 02 10:36:36.766317 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 02 10:36:36.766317 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:36.766317 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:36.766317 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 02 10:36:36.766317 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 02 10:36:36.766317 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 2581cb541a2441e2a318fc2657d7fb25. Aug 02 10:36:36.766317 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:36.766317 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 478/932] 10.222.0.115 () {60 vars in 1145 bytes} [Tue Aug 2 10:36:36 2022] POST /identity/v3/projects => generated 262 bytes in 304 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:36.766317 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78363b-1608-4dc1-aedb-24444fe4cddd None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:36.797735 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270b70b-caa5-46ed-b213-8d3a73c79044 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:36.843600 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e8b700-b6c6-4091-848d-e44db6e3fdb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:36.845166 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8b700-b6c6-4091-848d-e44db6e3fdb9 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:36.845671 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8b700-b6c6-4091-848d-e44db6e3fdb9 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:36.846276 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e8b700-b6c6-4091-848d-e44db6e3fdb9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:36.847383 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e8b700-b6c6-4091-848d-e44db6e3fdb9 None admin] RBAC: Adding query filter params (interface=public, service_id=d99d7fa2a67c4c19a4b73dc7faf99b46, region_id=RegionOne) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:36.847926 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e8b700-b6c6-4091-848d-e44db6e3fdb9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:36.850104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e8b700-b6c6-4091-848d-e44db6e3fdb9 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:36.853663 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:36.863713 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 455/933] 10.222.0.115 () {56 vars in 1256 bytes} [Tue Aug 2 10:36:36 2022] GET /identity/v3/endpoints?service_id=d99d7fa2a67c4c19a4b73dc7faf99b46&interface=public®ion_id=RegionOne => generated 195 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:36.865305 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e8b700-b6c6-4091-848d-e44db6e3fdb9 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:36.935187 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c78363b-1608-4dc1-aedb-24444fe4cddd None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:36.986432 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249040c2-dc6e-4a27-91db-2068ca5ee94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:36.987618 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249040c2-dc6e-4a27-91db-2068ca5ee94a None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:36.988127 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249040c2-dc6e-4a27-91db-2068ca5ee94a None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:36.988614 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249040c2-dc6e-4a27-91db-2068ca5ee94a None admin] PATH_INFO: `/v3/projects/service` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:36.997838 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249040c2-dc6e-4a27-91db-2068ca5ee94a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:36.999947 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249040c2-dc6e-4a27-91db-2068ca5ee94a None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:37.004380 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e8b700-b6c6-4091-848d-e44db6e3fdb9 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:37.015590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-249040c2-dc6e-4a27-91db-2068ca5ee94a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:37.015590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:37.015590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:37.015590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:37.015590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:37.015590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:37.015590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:37.015590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:37.015590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:37.015590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:37.015590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:37.015590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:37.015590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:36:37.015590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:36:37.015590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:36:37.016583 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:36:37.016583 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:37.016583 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:37.016583 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:37.016583 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:37.016583 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:37.016583 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:37.016583 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:37.016583 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:37.016583 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:37.016583 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:37.016583 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:37.016583 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:37.016583 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:37.016583 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:37.016583 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:37.016583 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:37.017531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:36:37.017531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:36:37.017531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:36:37.017531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:36:37.017531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:36:37.017531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:36:37.017531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:37.017531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:37.017994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 479/934] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:36:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:37.020013 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c8f7a-e594-4993-a432-46c341852df2 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:37.020520 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c8f7a-e594-4993-a432-46c341852df2 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:37.021034 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c8f7a-e594-4993-a432-46c341852df2 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:37.031198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 480/935] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:37 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:37.031198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c8f7a-e594-4993-a432-46c341852df2 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:37.071548 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c4d8c8-e792-40dc-b065-d2c258f9c616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:37.072092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4d8c8-e792-40dc-b065-d2c258f9c616 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:37.072371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4d8c8-e792-40dc-b065-d2c258f9c616 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:37.072649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c4d8c8-e792-40dc-b065-d2c258f9c616 None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:37.073385 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4d8c8-e792-40dc-b065-d2c258f9c616 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:37.074826 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c4d8c8-e792-40dc-b065-d2c258f9c616 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:37.091087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 456/936] 10.222.0.115 () {56 vars in 1092 bytes} [Tue Aug 2 10:36:36 2022] GET /identity/v3/services => generated 1504 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:36:37.091087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0989d00-3d2f-4d22-9310-46cd916b7c56 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:37.091087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0989d00-3d2f-4d22-9310-46cd916b7c56 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:37.091087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0989d00-3d2f-4d22-9310-46cd916b7c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:37.096179 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:37.122497 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-e0989d00-3d2f-4d22-9310-46cd916b7c56 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:37.123887 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-e0989d00-3d2f-4d22-9310-46cd916b7c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:37.224754 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c8f7a-e594-4993-a432-46c341852df2 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:37.254561 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:37.276397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 457/937] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:37 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:37.301121 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b4686-24e2-4115-8366-a52c139e3640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:37.302311 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b4686-24e2-4115-8366-a52c139e3640 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:37.302890 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b4686-24e2-4115-8366-a52c139e3640 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:37.303620 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b4686-24e2-4115-8366-a52c139e3640 None admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:37.304733 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0989d00-3d2f-4d22-9310-46cd916b7c56 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:37.305113 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b4686-24e2-4115-8366-a52c139e3640 None admin] RBAC: Adding query filter params (name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:37.305668 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b4686-24e2-4115-8366-a52c139e3640 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:37.308464 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b4686-24e2-4115-8366-a52c139e3640 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:37.338557 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 481/938] 10.222.0.115 () {56 vars in 1117 bytes} [Tue Aug 2 10:36:37 2022] GET /identity/v3/projects?name=service => generated 413 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:37.441187 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0989d00-3d2f-4d22-9310-46cd916b7c56 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:37.485025 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c4be3e-7d11-48a0-addb-6273505aee2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:37.485693 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4be3e-7d11-48a0-addb-6273505aee2c None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:37.485969 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4be3e-7d11-48a0-addb-6273505aee2c None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:37.486240 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c4be3e-7d11-48a0-addb-6273505aee2c None admin] PATH_INFO: `/v3/services/compute` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:37.486954 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c4be3e-7d11-48a0-addb-6273505aee2c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:37.488463 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c4be3e-7d11-48a0-addb-6273505aee2c None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:37.499276 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-39c4be3e-7d11-48a0-addb-6273505aee2c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 10:36:37.499276 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:37.499276 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:36:37.499276 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:36:37.499276 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:36:37.499276 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:36:37.499276 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:36:37.499276 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:37.499276 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 10:36:37.499276 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:37.499276 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:36:37.499276 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:37.499276 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:37.499276 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:37.499276 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:37.499276 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:37.499276 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:37.500375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:37.500375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:37.500375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:37.500375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:37.500375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:37.500375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:37.500375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:36:37.500375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:36:37.500375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:36:37.500375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:36:37.500375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:37.500375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:37.500375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:37.500375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:37.500375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:37.500375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:37.501228 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:37.501228 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:37.501228 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:37.501228 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:37.501228 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:37.501228 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:37.501228 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:37.501228 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:37.501228 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:37.501228 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:37.501228 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:36:37.501228 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:37.501228 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 10:36:37.501228 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:37.501228 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 458/939] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:36:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:37.507431 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b4686-24e2-4115-8366-a52c139e3640 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:37.665075 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:37.675367 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b4686-24e2-4115-8366-a52c139e3640 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:37.732478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67515142-1753-4f79-9f44-d558116e6de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:37.733129 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67515142-1753-4f79-9f44-d558116e6de7 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:37.733414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67515142-1753-4f79-9f44-d558116e6de7 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:37.733695 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67515142-1753-4f79-9f44-d558116e6de7 None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:37.734397 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67515142-1753-4f79-9f44-d558116e6de7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:37.734713 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67515142-1753-4f79-9f44-d558116e6de7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:37.736161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67515142-1753-4f79-9f44-d558116e6de7 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:37.744827 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 482/940] 10.222.0.115 () {56 vars in 1117 bytes} [Tue Aug 2 10:36:37 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:36:37.751463 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c4be3e-7d11-48a0-addb-6273505aee2c None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:37.821039 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:37.869643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c4be3e-7d11-48a0-addb-6273505aee2c None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:37.908169 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84778ec4-e08c-4c49-ac99-51e324673777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:37.908795 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84778ec4-e08c-4c49-ac99-51e324673777 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:37.909071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84778ec4-e08c-4c49-ac99-51e324673777 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:37.909341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84778ec4-e08c-4c49-ac99-51e324673777 None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:37.910036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84778ec4-e08c-4c49-ac99-51e324673777 None admin] RBAC: Adding query filter params (type=compute) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:37.910333 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84778ec4-e08c-4c49-ac99-51e324673777 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:37.911752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84778ec4-e08c-4c49-ac99-51e324673777 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:37.917377 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 459/941] 10.222.0.115 () {56 vars in 1117 bytes} [Tue Aug 2 10:36:37 2022] GET /identity/v3/services?type=compute => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:36:38.140901 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda426d-d04c-4ec5-bb0a-7b9c5f1887cc None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:38.141206 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda426d-d04c-4ec5-bb0a-7b9c5f1887cc None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:38.141532 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda426d-d04c-4ec5-bb0a-7b9c5f1887cc None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:38.143161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 483/942] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:38.148672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198d2a5-f8ff-45b0-a0a7-de92ed52c2a9 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:38.149062 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198d2a5-f8ff-45b0-a0a7-de92ed52c2a9 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:38.149239 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198d2a5-f8ff-45b0-a0a7-de92ed52c2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:38.170324 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-e198d2a5-f8ff-45b0-a0a7-de92ed52c2a9 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:38.173016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-e198d2a5-f8ff-45b0-a0a7-de92ed52c2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:38.252381 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 460/943] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:38 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:38.274770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda426d-d04c-4ec5-bb0a-7b9c5f1887cc None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:38.428641 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda426d-d04c-4ec5-bb0a-7b9c5f1887cc None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:38.469388 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7760e82-ab5d-45ce-8d55-9e30b9ef28ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:38.470128 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7760e82-ab5d-45ce-8d55-9e30b9ef28ae None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:38.470438 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7760e82-ab5d-45ce-8d55-9e30b9ef28ae None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:38.470711 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7760e82-ab5d-45ce-8d55-9e30b9ef28ae None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:38.484134 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7760e82-ab5d-45ce-8d55-9e30b9ef28ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:36:38.489239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-b7760e82-ab5d-45ce-8d55-9e30b9ef28ae None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:36:38.489239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:38.489239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:38.489239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:38.489239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:38.489239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:38.489239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:38.489239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:38.489239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:38.489239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:38.489239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:38.489239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:38.489239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:36:38.489239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:36:38.489239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:36:38.489239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:36:38.501086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:36:38.501086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:36:38.501086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:36:38.501086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:36:38.501086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:38.501086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:38.501086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:38.501086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:38.501086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:38.501086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:38.501086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:38.501086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:38.501086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:38.501086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:38.501086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:38.501086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:38.502445 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:38.502445 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:38.502445 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:38.502445 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:38.502445 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:36:38.502445 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:36:38.502445 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:36:38.502445 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:36:38.502445 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:36:38.502445 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:36:38.502445 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:36:38.502445 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:38.502445 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 484/944] 10.222.0.115 () {56 vars in 1125 bytes} [Tue Aug 2 10:36:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:38.503653 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e198d2a5-f8ff-45b0-a0a7-de92ed52c2a9 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:38.581055 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:38.642614 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e198d2a5-f8ff-45b0-a0a7-de92ed52c2a9 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:38.688649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473356e-7c53-49d8-b765-6b3636193594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:38.689191 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473356e-7c53-49d8-b765-6b3636193594 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:38.689467 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473356e-7c53-49d8-b765-6b3636193594 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:38.689736 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7473356e-7c53-49d8-b765-6b3636193594 None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:38.690460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7473356e-7c53-49d8-b765-6b3636193594 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:38.690758 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7473356e-7c53-49d8-b765-6b3636193594 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:38.692183 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7473356e-7c53-49d8-b765-6b3636193594 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:38.702956 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 461/945] 10.222.0.115 () {56 vars in 1120 bytes} [Tue Aug 2 10:36:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:38.712418 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7760e82-ab5d-45ce-8d55-9e30b9ef28ae None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:38.740190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:38.863486 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7760e82-ab5d-45ce-8d55-9e30b9ef28ae None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:38.908067 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868fb216-6c16-43bc-96bb-c4b1aeeb4f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:38.908733 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fb216-6c16-43bc-96bb-c4b1aeeb4f14 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:38.909048 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fb216-6c16-43bc-96bb-c4b1aeeb4f14 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:38.909257 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fb216-6c16-43bc-96bb-c4b1aeeb4f14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:38.921590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868fb216-6c16-43bc-96bb-c4b1aeeb4f14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:38.923221 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868fb216-6c16-43bc-96bb-c4b1aeeb4f14 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:38.943130 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-868fb216-6c16-43bc-96bb-c4b1aeeb4f14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:38.943130 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:38.943130 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:38.943130 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:38.943130 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:38.943130 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:38.943130 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:38.943130 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:38.943130 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:38.943130 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:38.943130 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:38.943130 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:38.943130 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:38.943130 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:38.943130 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:38.945979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:38.945979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:38.945979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:38.945979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:38.945979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:38.945979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:38.945979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:38.945979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:38.945979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:38.945979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:38.945979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:38.945979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:38.945979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:38.945979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:38.945979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:38.945979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:38.945979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:38.947957 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:38.947957 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:38.947957 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:38.947957 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:38.947957 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 485/946] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:38.953703 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473356e-7c53-49d8-b765-6b3636193594 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:39.123348 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7473356e-7c53-49d8-b765-6b3636193594 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:39.140626 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:39.165526 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ab2f3-e1ee-4a92-90c8-f42d4ff3e65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:39.166214 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ab2f3-e1ee-4a92-90c8-f42d4ff3e65f None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:39.166496 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ab2f3-e1ee-4a92-90c8-f42d4ff3e65f None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:39.166772 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754ab2f3-e1ee-4a92-90c8-f42d4ff3e65f None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:39.167501 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754ab2f3-e1ee-4a92-90c8-f42d4ff3e65f None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:39.167811 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754ab2f3-e1ee-4a92-90c8-f42d4ff3e65f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:39.169825 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754ab2f3-e1ee-4a92-90c8-f42d4ff3e65f None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:39.183669 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 462/947] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:38 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:39.190131 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868fb216-6c16-43bc-96bb-c4b1aeeb4f14 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:39.308572 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:39.356729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868fb216-6c16-43bc-96bb-c4b1aeeb4f14 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:39.402324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc80b5f-adb6-4f75-9764-c2b3cc544703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:39.403985 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc80b5f-adb6-4f75-9764-c2b3cc544703 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:39.404479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc80b5f-adb6-4f75-9764-c2b3cc544703 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:39.404982 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc80b5f-adb6-4f75-9764-c2b3cc544703 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:39.421896 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc80b5f-adb6-4f75-9764-c2b3cc544703 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:39.424678 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc80b5f-adb6-4f75-9764-c2b3cc544703 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:39.446831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-8bc80b5f-adb6-4f75-9764-c2b3cc544703 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:36:39.446831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:39.446831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:39.446831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:39.446831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:39.446831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:39.446831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:39.446831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:39.446831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:39.446831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:39.446831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:39.446831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:39.446831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:36:39.446831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:36:39.446831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:36:39.446831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:36:39.449225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:39.449225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:39.449225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:36:39.449225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:39.449225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:36:39.449225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:39.449225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:39.449225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:39.449225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:39.449225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:39.449225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:39.449225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:39.449225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:39.449225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:39.449225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:39.449225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:39.449225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:39.449225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:39.451351 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:39.451351 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:39.451351 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:36:39.451351 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:36:39.451351 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:36:39.451351 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:36:39.451351 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:36:39.451351 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:36:39.451351 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:36:39.451351 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:39.451351 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 486/948] 10.222.0.115 () {56 vars in 1122 bytes} [Tue Aug 2 10:36:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:36:39.463224 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ab2f3-e1ee-4a92-90c8-f42d4ff3e65f None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:39.613210 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754ab2f3-e1ee-4a92-90c8-f42d4ff3e65f None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:39.652430 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c629914-4548-4985-97bb-d4d22a0e029e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:39.653629 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c629914-4548-4985-97bb-d4d22a0e029e None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:39.654171 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c629914-4548-4985-97bb-d4d22a0e029e None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:39.654656 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c629914-4548-4985-97bb-d4d22a0e029e None admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:39.655865 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c629914-4548-4985-97bb-d4d22a0e029e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:39.656397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c629914-4548-4985-97bb-d4d22a0e029e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:39.658067 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c629914-4548-4985-97bb-d4d22a0e029e None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:39.679923 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 463/949] 10.222.0.115 () {56 vars in 1154 bytes} [Tue Aug 2 10:36:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:39.686457 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc80b5f-adb6-4f75-9764-c2b3cc544703 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:39.743501 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4dfa4-b9a1-4960-8177-02d2c46028eb None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:39.744000 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4dfa4-b9a1-4960-8177-02d2c46028eb None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:39.744493 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa4dfa4-b9a1-4960-8177-02d2c46028eb None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:39.751174 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 464/950] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:39 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:36:39.755861 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e653144-d519-4e87-836a-d7b0bbbe62db None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:39.756453 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e653144-d519-4e87-836a-d7b0bbbe62db None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:39.756968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e653144-d519-4e87-836a-d7b0bbbe62db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:39.777311 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-6e653144-d519-4e87-836a-d7b0bbbe62db None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:39.779370 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-6e653144-d519-4e87-836a-d7b0bbbe62db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:39.815237 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:39.850673 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc80b5f-adb6-4f75-9764-c2b3cc544703 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:39.891155 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 465/951] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:39 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:36:39.910396 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e653144-d519-4e87-836a-d7b0bbbe62db None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:39.916689 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:39.922649 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c07c0c-c9f6-4f78-87ad-cb0b22191fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:39.923636 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c07c0c-c9f6-4f78-87ad-cb0b22191fe0 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:39.924250 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c07c0c-c9f6-4f78-87ad-cb0b22191fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:39.924717 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c07c0c-c9f6-4f78-87ad-cb0b22191fe0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:39.940454 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c07c0c-c9f6-4f78-87ad-cb0b22191fe0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:39.942558 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c07c0c-c9f6-4f78-87ad-cb0b22191fe0 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:39.963423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-52c07c0c-c9f6-4f78-87ad-cb0b22191fe0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:39.963423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:39.963423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:39.963423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:39.963423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:39.963423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:39.963423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:39.963423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:39.963423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:39.963423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:39.963423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:39.963423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:39.963423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:39.963423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:39.963423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:39.964412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:39.964412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:39.964412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:39.964412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:39.964412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:39.964412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:39.964412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:39.964412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:39.964412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:39.964412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:39.964412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:39.964412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:39.964412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:39.964412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:39.964412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:39.964412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:39.964412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:39.966153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:39.966153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:39.966153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:39.966153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:39.969687 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 487/952] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:39.976078 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c07c0c-c9f6-4f78-87ad-cb0b22191fe0 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:40.077302 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e653144-d519-4e87-836a-d7b0bbbe62db None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:40.121271 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25102a2-46e5-4116-830f-6f22053f02d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:40.122860 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25102a2-46e5-4116-830f-6f22053f02d9 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:40.123277 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25102a2-46e5-4116-830f-6f22053f02d9 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:40.123652 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25102a2-46e5-4116-830f-6f22053f02d9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:40.124391 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25102a2-46e5-4116-830f-6f22053f02d9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:40.125274 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c07c0c-c9f6-4f78-87ad-cb0b22191fe0 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:40.126150 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25102a2-46e5-4116-830f-6f22053f02d9 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:40.136699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-c25102a2-46e5-4116-830f-6f22053f02d9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:36:40.136699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:40.136699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:36:40.136699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:36:40.136699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:36:40.136699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:36:40.136699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:36:40.136699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:40.136699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:36:40.136699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:40.136699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:36:40.136699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:40.136699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:40.136699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:40.136699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:40.136699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:40.136699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:40.139885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:40.139885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:40.139885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:40.139885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:40.139885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:40.139885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:40.139885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:36:40.139885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:36:40.139885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:36:40.139885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:36:40.139885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:40.139885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:40.139885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:40.139885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:40.139885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:40.139885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:40.141232 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:40.141232 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:40.141232 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:40.141232 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:40.141232 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:40.141232 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:40.141232 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:40.141232 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:40.141232 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:40.141232 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:40.141232 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:36:40.141232 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:40.141232 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:36:40.141232 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:40.144769 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 466/953] 10.222.0.115 () {56 vars in 1131 bytes} [Tue Aug 2 10:36:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:40.146317 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25102a2-46e5-4116-830f-6f22053f02d9 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:40.177289 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd2308f-59b8-48eb-86b7-9abd6bb7f484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:40.178403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2308f-59b8-48eb-86b7-9abd6bb7f484 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:40.178898 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2308f-59b8-48eb-86b7-9abd6bb7f484 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:40.179395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd2308f-59b8-48eb-86b7-9abd6bb7f484 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:40.180504 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd2308f-59b8-48eb-86b7-9abd6bb7f484 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:40.181075 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd2308f-59b8-48eb-86b7-9abd6bb7f484 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:40.182126 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:40.183068 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd2308f-59b8-48eb-86b7-9abd6bb7f484 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:40.198724 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 488/954] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:39 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:40.207785 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd2308f-59b8-48eb-86b7-9abd6bb7f484 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:40.266895 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:40.290917 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25102a2-46e5-4116-830f-6f22053f02d9 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:40.329535 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f95063-7eb6-46db-a47e-8c19b6f7bec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:40.330179 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f95063-7eb6-46db-a47e-8c19b6f7bec4 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:40.330527 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f95063-7eb6-46db-a47e-8c19b6f7bec4 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:40.331024 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f95063-7eb6-46db-a47e-8c19b6f7bec4 None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:40.331362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd2308f-59b8-48eb-86b7-9abd6bb7f484 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:40.331970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f95063-7eb6-46db-a47e-8c19b6f7bec4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:40.332111 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f95063-7eb6-46db-a47e-8c19b6f7bec4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:40.333452 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f95063-7eb6-46db-a47e-8c19b6f7bec4 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:40.344712 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 467/955] 10.222.0.115 () {56 vars in 1127 bytes} [Tue Aug 2 10:36:40 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:40.344712 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fbc082-148e-4f2a-89b4-1853865578fc None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:40.344712 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fbc082-148e-4f2a-89b4-1853865578fc None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:40.344712 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fbc082-148e-4f2a-89b4-1853865578fc None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:40.344712 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 468/956] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:40 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:40.345735 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fbc082-148e-4f2a-89b4-1853865578fc None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:40.379208 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44effd5-91bc-461e-a544-d105accdb3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:40.379851 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44effd5-91bc-461e-a544-d105accdb3c7 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:40.380131 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44effd5-91bc-461e-a544-d105accdb3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:40.380458 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44effd5-91bc-461e-a544-d105accdb3c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:40.388318 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44effd5-91bc-461e-a544-d105accdb3c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:40.389881 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44effd5-91bc-461e-a544-d105accdb3c7 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:40.405414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-e44effd5-91bc-461e-a544-d105accdb3c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:40.405414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:40.405414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:40.405414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:40.405414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:40.405414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:40.405414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:40.405414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:40.405414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:40.405414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:40.405414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:40.405414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:40.405414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:36:40.405414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:36:40.405414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:36:40.407428 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:36:40.407428 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:40.407428 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:40.407428 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:40.407428 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:40.407428 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:40.407428 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:40.407428 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:40.407428 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:40.407428 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:40.407428 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:40.407428 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:40.407428 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:40.407428 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:40.407428 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:40.407428 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:40.407428 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:40.409648 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:36:40.409648 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:36:40.409648 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:36:40.409648 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:36:40.409648 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:36:40.409648 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:36:40.409648 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:40.409648 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:40.410627 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 489/957] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:36:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:40.412393 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a0de3-19c0-4bdd-abb0-2a1fbec5b68b None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:40.412846 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a0de3-19c0-4bdd-abb0-2a1fbec5b68b None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:40.413283 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a0de3-19c0-4bdd-abb0-2a1fbec5b68b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:40.434034 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-fb1a0de3-19c0-4bdd-abb0-2a1fbec5b68b None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:40.436513 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-fb1a0de3-19c0-4bdd-abb0-2a1fbec5b68b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:40.480577 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:40.492753 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fbc082-148e-4f2a-89b4-1853865578fc None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:40.507478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 490/958] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:40 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:40.509151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56754b2-f7b8-4555-8fa3-601c54408d6a None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:40.509438 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56754b2-f7b8-4555-8fa3-601c54408d6a None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:40.509717 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56754b2-f7b8-4555-8fa3-601c54408d6a None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:40.511078 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 491/959] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:40.511827 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56754b2-f7b8-4555-8fa3-601c54408d6a None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:40.534268 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3603131-4a81-4636-8b70-21f81bb1b175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:40.534985 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3603131-4a81-4636-8b70-21f81bb1b175 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:40.535357 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3603131-4a81-4636-8b70-21f81bb1b175 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:40.535702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3603131-4a81-4636-8b70-21f81bb1b175 None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:40.536453 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3603131-4a81-4636-8b70-21f81bb1b175 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:40.536756 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3603131-4a81-4636-8b70-21f81bb1b175 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:40.538162 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3603131-4a81-4636-8b70-21f81bb1b175 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:40.543194 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 469/960] 10.222.0.115 () {56 vars in 1127 bytes} [Tue Aug 2 10:36:40 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:40.548697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f1e98-77dd-4114-b803-cbe46ecfbac9 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:40.548982 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f1e98-77dd-4114-b803-cbe46ecfbac9 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:40.549589 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49f1e98-77dd-4114-b803-cbe46ecfbac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:40.570227 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-d49f1e98-77dd-4114-b803-cbe46ecfbac9 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:40.571450 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-d49f1e98-77dd-4114-b803-cbe46ecfbac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:40.573017 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:40.653777 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56754b2-f7b8-4555-8fa3-601c54408d6a None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:40.663675 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 470/961] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:40 2022] POST /identity/v3/auth/tokens => generated 1476 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:40.664635 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f1e98-77dd-4114-b803-cbe46ecfbac9 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:40.698111 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2754213-e331-4272-a67d-3e8a5e461786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:40.698732 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2754213-e331-4272-a67d-3e8a5e461786 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:40.699015 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2754213-e331-4272-a67d-3e8a5e461786 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:40.699479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2754213-e331-4272-a67d-3e8a5e461786 None admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:40.700121 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2754213-e331-4272-a67d-3e8a5e461786 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:40.700437 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2754213-e331-4272-a67d-3e8a5e461786 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:40.702054 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2754213-e331-4272-a67d-3e8a5e461786 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:40.716160 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 492/962] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:36:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:40.717371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2754213-e331-4272-a67d-3e8a5e461786 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:40.767309 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:40.783633 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49f1e98-77dd-4114-b803-cbe46ecfbac9 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:40.835521 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b915824-0ee5-43b2-a805-254c4fe0f2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:40.836136 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b915824-0ee5-43b2-a805-254c4fe0f2d2 None admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:40.836417 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b915824-0ee5-43b2-a805-254c4fe0f2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:40.836554 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:40.836670 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b915824-0ee5-43b2-a805-254c4fe0f2d2 None admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:40.837340 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b915824-0ee5-43b2-a805-254c4fe0f2d2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:40.838758 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b915824-0ee5-43b2-a805-254c4fe0f2d2 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:40.841732 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: WARNING keystone.server.flask.common [None req-9b915824-0ee5-43b2-a805-254c4fe0f2d2 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 02 10:36:40.843515 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2754213-e331-4272-a67d-3e8a5e461786 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:40.892019 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56471ce3-89c2-4ead-8589-ace149b44386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:40.892642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56471ce3-89c2-4ead-8589-ace149b44386 None admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:40.892912 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56471ce3-89c2-4ead-8589-ace149b44386 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:40.893184 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56471ce3-89c2-4ead-8589-ace149b44386 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:40.893896 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56471ce3-89c2-4ead-8589-ace149b44386 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:40.895280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56471ce3-89c2-4ead-8589-ace149b44386 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:40.899822 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.sql.core [None req-9b915824-0ee5-43b2-a805-254c4fe0f2d2 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 02 10:36:40.899822 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 02 10:36:40.899822 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [parameters: {'user_id': 'fa63ebc2ffa34226b6181dda827b1309', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 02 10:36:40.899822 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=92111) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 02 10:36:40.918154 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-9b915824-0ee5-43b2-a805-254c4fe0f2d2 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 02 10:36:40.918154 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:40.918154 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 10:36:40.918154 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 10:36:40.918154 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 10:36:40.918154 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 10:36:40.918154 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 10:36:40.918154 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 10:36:40.918154 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 10:36:40.918154 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application conn.query(q) Aug 02 10:36:40.918154 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 10:36:40.918154 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 10:36:40.918154 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 10:36:40.918154 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application result.read() Aug 02 10:36:40.918154 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 10:36:40.919725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 10:36:40.919725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 10:36:40.919725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 10:36:40.919725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 10:36:40.919725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 10:36:40.919725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 10:36:40.919725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 10:36:40.919725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 02 10:36:40.919725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:40.919725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 02 10:36:40.919725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:40.919725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:40.919725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 02 10:36:40.919725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 02 10:36:40.919725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 02 10:36:40.919725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 02 10:36:40.919725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 10:36:40.919725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application next(self.gen) Aug 02 10:36:40.921476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 02 10:36:40.921476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application yield resource Aug 02 10:36:40.921476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 02 10:36:40.921476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application next(self.gen) Aug 02 10:36:40.921476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 02 10:36:40.921476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self.session.rollback() Aug 02 10:36:40.921476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 02 10:36:40.921476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self.force_reraise() Aug 02 10:36:40.921476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 02 10:36:40.921476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise self.value Aug 02 10:36:40.921476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 02 10:36:40.921476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 02 10:36:40.921476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 02 10:36:40.921476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application session.commit() Aug 02 10:36:40.921476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 02 10:36:40.921476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 02 10:36:40.921476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 02 10:36:40.923021 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self._prepare_impl() Aug 02 10:36:40.923021 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 02 10:36:40.923021 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self.session.flush() Aug 02 10:36:40.923021 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 02 10:36:40.923021 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self._flush(objects) Aug 02 10:36:40.923021 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 02 10:36:40.923021 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 02 10:36:40.923021 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 02 10:36:40.923021 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application compat.raise_( Aug 02 10:36:40.923021 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 10:36:40.923021 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception Aug 02 10:36:40.923021 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 02 10:36:40.923021 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application flush_context.execute() Aug 02 10:36:40.923021 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 02 10:36:40.923021 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rec.execute(self) Aug 02 10:36:40.923021 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 02 10:36:40.923021 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 02 10:36:40.924305 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 02 10:36:40.924305 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 02 10:36:40.924305 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 02 10:36:40.924305 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 02 10:36:40.924305 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Aug 02 10:36:40.924305 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 02 10:36:40.924305 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Aug 02 10:36:40.924305 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 02 10:36:40.924305 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Aug 02 10:36:40.924305 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 02 10:36:40.924305 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Aug 02 10:36:40.924305 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 02 10:36:40.924305 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Aug 02 10:36:40.924305 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 02 10:36:40.924305 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 02 10:36:40.924305 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception Aug 02 10:36:40.925176 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Aug 02 10:36:40.925176 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 02 10:36:40.925176 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 02 10:36:40.925176 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 02 10:36:40.925176 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 02 10:36:40.925176 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application result = self._query(query) Aug 02 10:36:40.925176 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 02 10:36:40.925176 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application conn.query(q) Aug 02 10:36:40.925176 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 02 10:36:40.925176 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 02 10:36:40.925176 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 02 10:36:40.925176 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application result.read() Aug 02 10:36:40.925176 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 02 10:36:40.925176 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 02 10:36:40.925176 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 02 10:36:40.925176 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 02 10:36:40.926663 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 02 10:36:40.926663 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 02 10:36:40.926663 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 02 10:36:40.926663 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 02 10:36:40.926663 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 02 10:36:40.926663 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 02 10:36:40.926663 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [parameters: {'user_id': 'fa63ebc2ffa34226b6181dda827b1309', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 02 10:36:40.926663 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 02 10:36:40.926663 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:40.926663 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:36:40.926663 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:40.926663 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:40.926663 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:40.926663 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:40.926663 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:40.927714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:40.927714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:40.927714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:40.927714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:40.927714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:40.927714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:40.927714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:40.927714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 02 10:36:40.927714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 02 10:36:40.927714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:40.927714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:40.927714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:36:40.927714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:40.927714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:36:40.927714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:40.927714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 02 10:36:40.927714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 02 10:36:40.928658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 02 10:36:40.928658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 02 10:36:40.928658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 02 10:36:40.928658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 02 10:36:40.928658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 02 10:36:40.928658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:40.928658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 471/963] 10.222.0.115 () {60 vars in 1136 bytes} [Tue Aug 2 10:36:40 2022] POST /identity/v3/users => generated 160 bytes in 254 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:40.928658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b915824-0ee5-43b2-a805-254c4fe0f2d2 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:40.975541 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 493/964] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:36:40 2022] POST /identity/v3/endpoints => generated 351 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:36:40.976302 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56471ce3-89c2-4ead-8589-ace149b44386 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:41.063728 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b915824-0ee5-43b2-a805-254c4fe0f2d2 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:41.123412 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f6a45-03a3-43bc-b495-b5e386ca2ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:41.124171 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f6a45-03a3-43bc-b495-b5e386ca2ad0 None admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:41.124449 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f6a45-03a3-43bc-b495-b5e386ca2ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:41.124807 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983f6a45-03a3-43bc-b495-b5e386ca2ad0 None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:41.125552 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983f6a45-03a3-43bc-b495-b5e386ca2ad0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:41.127874 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983f6a45-03a3-43bc-b495-b5e386ca2ad0 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:41.140752 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56471ce3-89c2-4ead-8589-ace149b44386 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:41.193148 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3735aa-c2ab-405d-9e4d-fb3169252b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:41.193148 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3735aa-c2ab-405d-9e4d-fb3169252b17 None admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:41.193148 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3735aa-c2ab-405d-9e4d-fb3169252b17 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:41.193148 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3735aa-c2ab-405d-9e4d-fb3169252b17 None admin] PATH_INFO: `/v3/projects/c1eae881e5714bf4bd223809e1a2d920/users/2b03263937cb4ca7807a24da44d37b0f/roles/7fee85e1d15545a1926ac22137e06585` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:41.193869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3735aa-c2ab-405d-9e4d-fb3169252b17 None admin] RBAC: Authorizing `identity:create_grant(project_id=c1eae881e5714bf4bd223809e1a2d920, user_id=2b03263937cb4ca7807a24da44d37b0f, role_id=7fee85e1d15545a1926ac22137e06585)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:41.196258 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3735aa-c2ab-405d-9e4d-fb3169252b17 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:41.221174 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 472/965] 10.222.0.115 () {60 vars in 1146 bytes} [Tue Aug 2 10:36:40 2022] POST /identity/v3/services => generated 243 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:36:41.222136 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f6a45-03a3-43bc-b495-b5e386ca2ad0 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:41.232885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:41.291086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 494/966] 10.222.0.115 () {58 vars in 1444 bytes} [Tue Aug 2 10:36:40 2022] PUT /identity/v3/projects/c1eae881e5714bf4bd223809e1a2d920/users/2b03263937cb4ca7807a24da44d37b0f/roles/7fee85e1d15545a1926ac22137e06585 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:36:41.432436 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983f6a45-03a3-43bc-b495-b5e386ca2ad0 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:41.495289 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae2f3e-4b27-4763-a5a7-17f10621332b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:41.496209 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae2f3e-4b27-4763-a5a7-17f10621332b None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:41.497147 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae2f3e-4b27-4763-a5a7-17f10621332b None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:41.497659 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae2f3e-4b27-4763-a5a7-17f10621332b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:41.506933 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae2f3e-4b27-4763-a5a7-17f10621332b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:41.508883 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae2f3e-4b27-4763-a5a7-17f10621332b None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:41.528461 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-89ae2f3e-4b27-4763-a5a7-17f10621332b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 10:36:41.528461 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:41.528461 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:41.528461 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:41.528461 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:41.528461 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:41.528461 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:41.528461 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:41.528461 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:41.528461 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:41.528461 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:41.528461 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:41.528461 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:36:41.528461 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:36:41.528461 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:36:41.528461 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:36:41.533708 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:41.533708 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:41.533708 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:36:41.533708 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:41.533708 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:36:41.533708 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:41.533708 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:41.533708 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:41.533708 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:41.533708 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:41.533708 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:41.533708 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:41.533708 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:41.533708 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:41.533708 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:41.533708 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:41.533708 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:41.533708 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:41.534930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:41.534930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:41.534930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:36:41.534930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:36:41.534930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:36:41.534930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:36:41.534930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:36:41.534930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:36:41.534930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 02 10:36:41.534930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:41.534930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 473/967] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:36:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:41.537673 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3735aa-c2ab-405d-9e4d-fb3169252b17 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:41.551847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:41.724639 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3735aa-c2ab-405d-9e4d-fb3169252b17 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:41.770261 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a692e-e21e-42e2-9056-4a3ae65729f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:41.770904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a692e-e21e-42e2-9056-4a3ae65729f4 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:41.771221 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a692e-e21e-42e2-9056-4a3ae65729f4 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:41.771501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a692e-e21e-42e2-9056-4a3ae65729f4 None admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:41.773582 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0a692e-e21e-42e2-9056-4a3ae65729f4 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:41.773582 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0a692e-e21e-42e2-9056-4a3ae65729f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:41.774151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0a692e-e21e-42e2-9056-4a3ae65729f4 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:41.817895 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 495/968] 10.222.0.115 () {56 vars in 1106 bytes} [Tue Aug 2 10:36:41 2022] GET /identity/v3/users?name=cinder => generated 356 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:43.218049 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba12218-9b34-46c1-98a7-5ce9e6e4c1b6 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:43.218804 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba12218-9b34-46c1-98a7-5ce9e6e4c1b6 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:43.218804 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba12218-9b34-46c1-98a7-5ce9e6e4c1b6 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:43.220158 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 474/969] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:43.225270 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70d507-6d3f-4128-8b24-5ef9d46d93df None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:43.225674 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70d507-6d3f-4128-8b24-5ef9d46d93df None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:43.225841 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe70d507-6d3f-4128-8b24-5ef9d46d93df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:43.256304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-fe70d507-6d3f-4128-8b24-5ef9d46d93df None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:43.257536 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-fe70d507-6d3f-4128-8b24-5ef9d46d93df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:43.345059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:43.351472 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 496/970] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:43 2022] POST /identity/v3/auth/tokens => generated 1572 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:43.373667 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba12218-9b34-46c1-98a7-5ce9e6e4c1b6 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:43.441296 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:43.488280 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba12218-9b34-46c1-98a7-5ce9e6e4c1b6 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:43.509733 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02127da-0702-4e50-b93b-ee8a8a262412 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:43.510075 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02127da-0702-4e50-b93b-ee8a8a262412 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:43.510309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02127da-0702-4e50-b93b-ee8a8a262412 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:43.511933 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 497/971] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:36:43.517950 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743c0fc-1e93-41f5-afbb-d15855beb19b None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:43.518488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743c0fc-1e93-41f5-afbb-d15855beb19b None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:43.518994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1743c0fc-1e93-41f5-afbb-d15855beb19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:43.528990 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffd3f40-3d32-416d-93d1-bd2c65449466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:43.532429 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd3f40-3d32-416d-93d1-bd2c65449466 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:43.532429 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd3f40-3d32-416d-93d1-bd2c65449466 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:43.532429 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd3f40-3d32-416d-93d1-bd2c65449466 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:43.532429 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd3f40-3d32-416d-93d1-bd2c65449466 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:43.532429 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffd3f40-3d32-416d-93d1-bd2c65449466 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:43.538597 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-1743c0fc-1e93-41f5-afbb-d15855beb19b None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:43.539808 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-1743c0fc-1e93-41f5-afbb-d15855beb19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:43.541938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-9ffd3f40-3d32-416d-93d1-bd2c65449466 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:36:43.541938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:43.541938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:36:43.541938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:36:43.541938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:36:43.541938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:36:43.541938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:36:43.541938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:43.541938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:36:43.541938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:43.541938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:36:43.541938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:43.541938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:43.541938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:43.541938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:43.541938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:43.541938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:43.542941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:43.542941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:43.542941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:43.542941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:43.542941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:43.542941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:43.542941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:36:43.542941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:36:43.542941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:36:43.542941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:36:43.542941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:43.542941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:43.542941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:43.542941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:43.542941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:43.542941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:43.544494 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:43.544494 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:43.544494 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:43.544494 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:43.544494 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:43.544494 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:43.544494 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:43.544494 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:43.544494 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:43.544494 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:43.544494 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:36:43.544494 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:43.544494 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:36:43.544494 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:43.544494 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 475/972] 10.222.0.115 () {56 vars in 1131 bytes} [Tue Aug 2 10:36:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:43.546729 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da511b-dda1-454b-ab68-bf077260e6d7 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:43.546729 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da511b-dda1-454b-ab68-bf077260e6d7 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:43.546729 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32da511b-dda1-454b-ab68-bf077260e6d7 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:43.547895 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 476/973] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:43 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:43.550007 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32da511b-dda1-454b-ab68-bf077260e6d7 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:43.615419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 498/974] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:43 2022] POST /identity/v3/auth/tokens => generated 1572 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:43.616881 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d708c71-eee3-49b6-94d6-e322e55563ac None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:43.617042 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d708c71-eee3-49b6-94d6-e322e55563ac None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:43.617276 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d708c71-eee3-49b6-94d6-e322e55563ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:43.638487 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-6d708c71-eee3-49b6-94d6-e322e55563ac None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:43.639663 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-6d708c71-eee3-49b6-94d6-e322e55563ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:43.659814 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32da511b-dda1-454b-ab68-bf077260e6d7 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:43.683187 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:43.695304 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a34d1-1351-445a-a8b2-100173466fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:43.695921 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a34d1-1351-445a-a8b2-100173466fd6 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:43.696198 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a34d1-1351-445a-a8b2-100173466fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:43.696470 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1a34d1-1351-445a-a8b2-100173466fd6 None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:43.697152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a34d1-1351-445a-a8b2-100173466fd6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:43.697449 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a34d1-1351-445a-a8b2-100173466fd6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:43.698826 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1a34d1-1351-445a-a8b2-100173466fd6 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:43.704468 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 477/975] 10.222.0.115 () {56 vars in 1127 bytes} [Tue Aug 2 10:36:43 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:36:43.705282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a34d1-1351-445a-a8b2-100173466fd6 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:43.723391 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 499/976] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:43 2022] POST /identity/v3/auth/tokens => generated 1572 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:43.724230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d708c71-eee3-49b6-94d6-e322e55563ac None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:43.792388 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:43.811452 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1a34d1-1351-445a-a8b2-100173466fd6 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:43.845493 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d708c71-eee3-49b6-94d6-e322e55563ac None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:43.846944 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c226682-82c8-4353-9515-12b23477f4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:43.847630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c226682-82c8-4353-9515-12b23477f4da None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:43.847909 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c226682-82c8-4353-9515-12b23477f4da None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:43.848183 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c226682-82c8-4353-9515-12b23477f4da None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:43.859675 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c226682-82c8-4353-9515-12b23477f4da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:36:43.866298 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-6c226682-82c8-4353-9515-12b23477f4da None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:36:43.866298 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:43.866298 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:43.866298 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:43.866298 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:43.866298 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:43.866298 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:43.866298 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:43.866298 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:43.866298 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:43.866298 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:43.866298 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:43.866298 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:36:43.866298 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:36:43.866298 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:36:43.866298 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:36:43.867555 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:36:43.867555 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:36:43.867555 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:36:43.867555 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:36:43.867555 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:43.867555 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:43.867555 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:43.867555 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:43.867555 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:43.867555 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:43.867555 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:43.867555 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:43.867555 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:43.867555 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:43.867555 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:43.867555 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:43.868666 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:43.868666 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:43.868666 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:43.868666 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:43.868666 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:36:43.868666 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:36:43.868666 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:36:43.868666 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:36:43.868666 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:36:43.868666 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:36:43.868666 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:36:43.868666 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:43.868666 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 478/977] 10.222.0.115 () {56 vars in 1125 bytes} [Tue Aug 2 10:36:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:43.868666 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c226682-82c8-4353-9515-12b23477f4da None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:43.882654 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af41e92-8959-4573-bdfc-b59a33562918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:43.883495 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af41e92-8959-4573-bdfc-b59a33562918 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:43.883929 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af41e92-8959-4573-bdfc-b59a33562918 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:43.884360 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af41e92-8959-4573-bdfc-b59a33562918 None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:43.885298 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af41e92-8959-4573-bdfc-b59a33562918 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:43.885742 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af41e92-8959-4573-bdfc-b59a33562918 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:43.888385 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af41e92-8959-4573-bdfc-b59a33562918 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:43.900847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 500/978] 10.222.0.115 () {56 vars in 1127 bytes} [Tue Aug 2 10:36:43 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:43.900847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af41e92-8959-4573-bdfc-b59a33562918 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:43.925935 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:43.982208 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c226682-82c8-4353-9515-12b23477f4da None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:44.006467 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af41e92-8959-4573-bdfc-b59a33562918 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:44.018593 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c398b-2a23-419f-b181-cff17d498ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:44.019368 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c398b-2a23-419f-b181-cff17d498ec7 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:44.019646 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c398b-2a23-419f-b181-cff17d498ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:44.019918 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7c398b-2a23-419f-b181-cff17d498ec7 None admin] PATH_INFO: `/v3/services/compute` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:44.020542 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7c398b-2a23-419f-b181-cff17d498ec7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:44.021910 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7c398b-2a23-419f-b181-cff17d498ec7 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:44.031616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-8b7c398b-2a23-419f-b181-cff17d498ec7 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 10:36:44.031616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:44.031616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:36:44.031616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:36:44.031616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:36:44.031616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:36:44.031616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:36:44.031616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:44.031616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 10:36:44.031616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:44.031616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:36:44.031616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:44.031616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:44.031616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:44.031616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:44.031616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:44.031616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:44.032717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:44.032717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:44.032717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:44.032717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:44.032717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:44.032717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:44.032717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:36:44.032717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:36:44.032717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:36:44.032717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:36:44.032717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:44.032717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:44.032717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:44.032717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:44.032717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:44.032717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:44.033770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:44.033770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:44.033770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:44.033770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:44.033770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:44.033770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:44.033770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:44.033770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:44.033770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:44.033770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:44.033770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:36:44.033770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:44.033770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 10:36:44.033770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:44.034726 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:44.034803 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 479/979] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:36:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:44.034803 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c398b-2a23-419f-b181-cff17d498ec7 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:44.049138 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018873cc-df8a-495b-b312-6869f3c4449b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:44.049138 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018873cc-df8a-495b-b312-6869f3c4449b None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:44.049138 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018873cc-df8a-495b-b312-6869f3c4449b None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:44.049138 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018873cc-df8a-495b-b312-6869f3c4449b None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:44.049630 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018873cc-df8a-495b-b312-6869f3c4449b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:44.049630 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018873cc-df8a-495b-b312-6869f3c4449b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:44.049630 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018873cc-df8a-495b-b312-6869f3c4449b None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:44.057460 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 501/980] 10.222.0.115 () {56 vars in 1120 bytes} [Tue Aug 2 10:36:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:44.058286 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018873cc-df8a-495b-b312-6869f3c4449b None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:44.150717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7c398b-2a23-419f-b181-cff17d498ec7 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:44.162181 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:44.165946 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018873cc-df8a-495b-b312-6869f3c4449b None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:44.188119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9288b9-3ee1-4761-89e6-b5b73e9684b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:44.188817 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9288b9-3ee1-4761-89e6-b5b73e9684b5 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:44.189005 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9288b9-3ee1-4761-89e6-b5b73e9684b5 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:44.189250 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9288b9-3ee1-4761-89e6-b5b73e9684b5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:44.190116 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9288b9-3ee1-4761-89e6-b5b73e9684b5 None admin] RBAC: Adding query filter params (interface=admin, service_id=d99d7fa2a67c4c19a4b73dc7faf99b46, region_id=RegionOne) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:44.190823 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9288b9-3ee1-4761-89e6-b5b73e9684b5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:44.192712 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9288b9-3ee1-4761-89e6-b5b73e9684b5 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:44.202997 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 480/981] 10.222.0.115 () {56 vars in 1254 bytes} [Tue Aug 2 10:36:44 2022] GET /identity/v3/endpoints?service_id=d99d7fa2a67c4c19a4b73dc7faf99b46&interface=admin®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:44.204103 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9288b9-3ee1-4761-89e6-b5b73e9684b5 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:44.213109 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd79418-ab1e-4c77-b049-b101c843d8c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:44.213740 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd79418-ab1e-4c77-b049-b101c843d8c7 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:44.214024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd79418-ab1e-4c77-b049-b101c843d8c7 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:44.214300 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd79418-ab1e-4c77-b049-b101c843d8c7 None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:44.214990 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd79418-ab1e-4c77-b049-b101c843d8c7 None admin] RBAC: Adding query filter params (name=compute) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:44.215350 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd79418-ab1e-4c77-b049-b101c843d8c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:44.216730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd79418-ab1e-4c77-b049-b101c843d8c7 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:44.223632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 502/982] 10.222.0.115 () {56 vars in 1117 bytes} [Tue Aug 2 10:36:44 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:44.226510 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd79418-ab1e-4c77-b049-b101c843d8c7 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:44.281436 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:44.313763 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9288b9-3ee1-4761-89e6-b5b73e9684b5 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:44.338703 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd79418-ab1e-4c77-b049-b101c843d8c7 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:44.350039 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c47b8-9c37-4ade-b3ce-9a9ffe3ca0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:44.350674 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c47b8-9c37-4ade-b3ce-9a9ffe3ca0a0 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:44.350952 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c47b8-9c37-4ade-b3ce-9a9ffe3ca0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:44.351259 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355c47b8-9c37-4ade-b3ce-9a9ffe3ca0a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:44.362351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355c47b8-9c37-4ade-b3ce-9a9ffe3ca0a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:44.363844 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355c47b8-9c37-4ade-b3ce-9a9ffe3ca0a0 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:44.375779 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3fa90f-bf49-4dde-b93e-e30162bc44eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:44.376398 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fa90f-bf49-4dde-b93e-e30162bc44eb None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:44.376677 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fa90f-bf49-4dde-b93e-e30162bc44eb None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:44.376950 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3fa90f-bf49-4dde-b93e-e30162bc44eb None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:44.377599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3fa90f-bf49-4dde-b93e-e30162bc44eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:44.378950 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3fa90f-bf49-4dde-b93e-e30162bc44eb None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:44.379809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-355c47b8-9c37-4ade-b3ce-9a9ffe3ca0a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:44.379809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:44.379809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:44.379809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:44.379809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:44.379809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:44.379809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:44.379809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:44.379809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:44.379809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:44.379809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:44.379809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:44.379809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:44.379809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:44.379809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:44.380859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:44.380859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:44.380859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:44.380859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:44.380859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:44.380859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:44.380859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:44.380859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:44.380859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:44.380859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:44.380859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:44.380859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:44.380859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:44.380859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:44.380859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:44.380859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:44.380859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:44.386760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:44.386760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:44.386760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:44.386760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:44.386760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 481/983] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:44.386760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c47b8-9c37-4ade-b3ce-9a9ffe3ca0a0 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:44.387197 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 503/984] 10.222.0.115 () {56 vars in 1092 bytes} [Tue Aug 2 10:36:44 2022] GET /identity/v3/services => generated 1735 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:36:44.387503 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3fa90f-bf49-4dde-b93e-e30162bc44eb None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:44.423507 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:44.532105 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355c47b8-9c37-4ade-b3ce-9a9ffe3ca0a0 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:44.534378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3fa90f-bf49-4dde-b93e-e30162bc44eb None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:44.553976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:44.583519 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b47ea0-f257-4703-b727-1fb3006a71b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:44.583934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0f6f5-36cc-47c9-91b8-3f81d8feef92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:44.584068 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b47ea0-f257-4703-b727-1fb3006a71b0 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:44.584312 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b47ea0-f257-4703-b727-1fb3006a71b0 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:44.584583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b47ea0-f257-4703-b727-1fb3006a71b0 None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:44.584684 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0f6f5-36cc-47c9-91b8-3f81d8feef92 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:44.584879 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0f6f5-36cc-47c9-91b8-3f81d8feef92 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:44.585324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b0f6f5-36cc-47c9-91b8-3f81d8feef92 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:44.585431 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b47ea0-f257-4703-b727-1fb3006a71b0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:44.585702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b47ea0-f257-4703-b727-1fb3006a71b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:44.586499 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0f6f5-36cc-47c9-91b8-3f81d8feef92 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:44.586801 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0f6f5-36cc-47c9-91b8-3f81d8feef92 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:44.587565 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b47ea0-f257-4703-b727-1fb3006a71b0 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:44.588212 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b0f6f5-36cc-47c9-91b8-3f81d8feef92 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:44.596625 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 482/985] 10.222.0.115 () {56 vars in 1117 bytes} [Tue Aug 2 10:36:44 2022] GET /identity/v3/services?type=compute => generated 354 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:44.606356 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b47ea0-f257-4703-b727-1fb3006a71b0 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:44.607156 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 504/986] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:44 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:36:44.610946 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0f6f5-36cc-47c9-91b8-3f81d8feef92 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:44.724538 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:44.736773 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b47ea0-f257-4703-b727-1fb3006a71b0 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:44.756663 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b0f6f5-36cc-47c9-91b8-3f81d8feef92 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:44.776556 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b1bf5-9e01-4fc4-8467-9b220db4b28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:44.777210 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b1bf5-9e01-4fc4-8467-9b220db4b28f None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:44.777503 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b1bf5-9e01-4fc4-8467-9b220db4b28f None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:44.777898 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166b1bf5-9e01-4fc4-8467-9b220db4b28f None admin] PATH_INFO: `/v3/endpoints` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:44.779272 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b1bf5-9e01-4fc4-8467-9b220db4b28f None admin] RBAC: Adding query filter params (interface=public, service_id=2c051b7c4ac3431e9c756135d0501ab6, region_id=RegionOne) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:44.779581 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b1bf5-9e01-4fc4-8467-9b220db4b28f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:44.780965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166b1bf5-9e01-4fc4-8467-9b220db4b28f None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:44.787232 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 483/987] 10.222.0.115 () {56 vars in 1256 bytes} [Tue Aug 2 10:36:44 2022] GET /identity/v3/endpoints?service_id=2c051b7c4ac3431e9c756135d0501ab6&interface=public®ion_id=RegionOne => generated 195 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:44.792701 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b1bf5-9e01-4fc4-8467-9b220db4b28f None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:44.811343 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c68caf-908d-45e8-b6f5-1a5af5a28d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:44.812006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c68caf-908d-45e8-b6f5-1a5af5a28d7a None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:44.812289 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c68caf-908d-45e8-b6f5-1a5af5a28d7a None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:44.812565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c68caf-908d-45e8-b6f5-1a5af5a28d7a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:44.820673 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c68caf-908d-45e8-b6f5-1a5af5a28d7a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:44.822139 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c68caf-908d-45e8-b6f5-1a5af5a28d7a None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:44.836491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-93c68caf-908d-45e8-b6f5-1a5af5a28d7a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:36:44.836491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:44.836491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:44.836491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:44.836491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:44.836491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:44.836491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:44.836491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:44.836491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:44.836491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:44.836491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:44.836491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:44.836491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:36:44.836491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:36:44.836491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:36:44.836491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:36:44.837743 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:44.837743 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:44.837743 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:36:44.837743 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:44.837743 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:36:44.837743 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:44.837743 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:44.837743 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:44.837743 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:44.837743 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:44.837743 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:44.837743 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:44.837743 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:44.837743 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:44.837743 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:44.837743 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:44.837743 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:44.837743 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:44.839110 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:44.839110 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:44.839110 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:36:44.839110 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:36:44.839110 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:36:44.839110 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:36:44.839110 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:36:44.839110 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:36:44.839110 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 02 10:36:44.839110 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:44.843216 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 505/988] 10.222.0.115 () {56 vars in 1122 bytes} [Tue Aug 2 10:36:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:36:44.844151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c68caf-908d-45e8-b6f5-1a5af5a28d7a None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:44.880657 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:44.943787 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166b1bf5-9e01-4fc4-8467-9b220db4b28f None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:44.969949 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c68caf-908d-45e8-b6f5-1a5af5a28d7a None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:44.993844 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8c36c-1018-456c-860f-93cc7f1efd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:44.994464 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8c36c-1018-456c-860f-93cc7f1efd7f None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:44.994743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8c36c-1018-456c-860f-93cc7f1efd7f None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:44.995016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b8c36c-1018-456c-860f-93cc7f1efd7f None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:44.995704 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b8c36c-1018-456c-860f-93cc7f1efd7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:44.997191 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b8c36c-1018-456c-860f-93cc7f1efd7f None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:45.004598 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 484/989] 10.222.0.115 () {56 vars in 1092 bytes} [Tue Aug 2 10:36:44 2022] GET /identity/v3/services => generated 1735 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:36:45.027371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ee025-120b-47ab-94a8-2e87946651ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:45.028709 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ee025-120b-47ab-94a8-2e87946651ae None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:45.029056 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ee025-120b-47ab-94a8-2e87946651ae None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:45.029397 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ee025-120b-47ab-94a8-2e87946651ae None admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:45.030298 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ee025-120b-47ab-94a8-2e87946651ae None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:45.030675 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ee025-120b-47ab-94a8-2e87946651ae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:45.032314 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ee025-120b-47ab-94a8-2e87946651ae None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:45.054419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 506/990] 10.222.0.115 () {56 vars in 1154 bytes} [Tue Aug 2 10:36:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 386 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:45.061847 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8c36c-1018-456c-860f-93cc7f1efd7f None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:45.085382 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:45.189090 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b8c36c-1018-456c-860f-93cc7f1efd7f None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:45.234178 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39a1e3-5c77-4b93-9aff-39d7fa71d04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:45.235164 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39a1e3-5c77-4b93-9aff-39d7fa71d04b None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:45.235505 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39a1e3-5c77-4b93-9aff-39d7fa71d04b None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:45.236279 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39a1e3-5c77-4b93-9aff-39d7fa71d04b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:45.318674 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39a1e3-5c77-4b93-9aff-39d7fa71d04b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:45.320748 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39a1e3-5c77-4b93-9aff-39d7fa71d04b None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:45.342557 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:45.443655 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 485/991] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:36:45 2022] GET /identity/v3/auth/tokens => generated 1572 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:36:45.449258 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ee025-120b-47ab-94a8-2e87946651ae None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:45.556817 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:45.567300 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ee025-120b-47ab-94a8-2e87946651ae None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:45.606812 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d62f4-372a-41ef-b2e0-044ea7604071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:45.607515 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d62f4-372a-41ef-b2e0-044ea7604071 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:45.607800 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d62f4-372a-41ef-b2e0-044ea7604071 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:45.608077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d62f4-372a-41ef-b2e0-044ea7604071 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:45.620977 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d62f4-372a-41ef-b2e0-044ea7604071 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:45.623438 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d62f4-372a-41ef-b2e0-044ea7604071 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:45.641490 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-289d62f4-372a-41ef-b2e0-044ea7604071 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:45.641490 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:45.641490 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:45.641490 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:45.641490 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:45.641490 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:45.641490 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:45.641490 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:45.641490 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:45.641490 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:45.641490 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:45.641490 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:45.641490 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:45.641490 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:45.641490 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:45.642501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:45.642501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:45.642501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:45.642501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:45.642501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:45.642501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:45.642501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:45.642501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:45.642501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:45.642501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:45.642501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:45.642501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:45.642501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:45.642501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:45.642501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:45.642501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:45.642501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:45.644233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:45.644233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:45.644233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:45.644233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:45.647398 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 507/992] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:36:45.648776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39a1e3-5c77-4b93-9aff-39d7fa71d04b None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:45.777281 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39a1e3-5c77-4b93-9aff-39d7fa71d04b None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:45.803258 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:45.822735 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857e2bd-e81d-46da-8881-f9ed8125b6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:45.823407 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857e2bd-e81d-46da-8881-f9ed8125b6a7 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:45.823691 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857e2bd-e81d-46da-8881-f9ed8125b6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:45.823991 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857e2bd-e81d-46da-8881-f9ed8125b6a7 None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:45.825232 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857e2bd-e81d-46da-8881-f9ed8125b6a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:45.825584 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857e2bd-e81d-46da-8881-f9ed8125b6a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:45.826945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e857e2bd-e81d-46da-8881-f9ed8125b6a7 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:45.840668 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 486/993] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:45.846742 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d62f4-372a-41ef-b2e0-044ea7604071 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:45.974108 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d62f4-372a-41ef-b2e0-044ea7604071 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:46.021629 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc89df0-ed40-4567-b44a-29c36a6e7648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:46.022311 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc89df0-ed40-4567-b44a-29c36a6e7648 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:46.022597 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc89df0-ed40-4567-b44a-29c36a6e7648 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:46.022873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc89df0-ed40-4567-b44a-29c36a6e7648 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:46.031616 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc89df0-ed40-4567-b44a-29c36a6e7648 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:46.032987 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc89df0-ed40-4567-b44a-29c36a6e7648 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:46.045759 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-ddc89df0-ed40-4567-b44a-29c36a6e7648 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:46.045759 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:46.045759 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:46.045759 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:46.045759 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:46.045759 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:46.045759 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:46.045759 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:46.045759 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:46.045759 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:46.045759 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:46.045759 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:46.045759 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:36:46.045759 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:36:46.045759 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:36:46.046770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:36:46.046770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:46.046770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:46.046770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:46.046770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:46.046770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:46.046770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:46.046770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:46.046770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:46.046770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:46.046770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:46.046770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:46.046770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:46.046770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:46.046770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:46.046770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:46.046770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:46.048050 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:36:46.048050 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:36:46.048050 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:36:46.048050 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:36:46.048050 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:36:46.048050 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:36:46.048050 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:46.048050 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:46.051495 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 508/994] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:36:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:36:46.052828 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857e2bd-e81d-46da-8881-f9ed8125b6a7 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:46.167459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e857e2bd-e81d-46da-8881-f9ed8125b6a7 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:46.220662 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac098c-5307-46de-8dbb-4480b7a0031e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:46.221682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac098c-5307-46de-8dbb-4480b7a0031e None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:46.221961 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac098c-5307-46de-8dbb-4480b7a0031e None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:46.222309 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac098c-5307-46de-8dbb-4480b7a0031e None admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:46.223598 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac098c-5307-46de-8dbb-4480b7a0031e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:46.223904 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac098c-5307-46de-8dbb-4480b7a0031e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:46.227294 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ac098c-5307-46de-8dbb-4480b7a0031e None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:46.247302 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 487/995] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:36:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:46.248569 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc89df0-ed40-4567-b44a-29c36a6e7648 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:46.251718 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:46.396175 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc89df0-ed40-4567-b44a-29c36a6e7648 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:46.448253 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e89c688-f7b2-4731-a4d2-126db51ea091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:46.448870 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e89c688-f7b2-4731-a4d2-126db51ea091 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:46.449153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e89c688-f7b2-4731-a4d2-126db51ea091 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:46.449965 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e89c688-f7b2-4731-a4d2-126db51ea091 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:46.450775 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e89c688-f7b2-4731-a4d2-126db51ea091 None admin] RBAC: Adding query filter params (role.id=7fee85e1d15545a1926ac22137e06585, scope.project.id=c1eae881e5714bf4bd223809e1a2d920, user.id=2b03263937cb4ca7807a24da44d37b0f) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:46.454844 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e89c688-f7b2-4731-a4d2-126db51ea091 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:46.454844 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e89c688-f7b2-4731-a4d2-126db51ea091 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:46.471078 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 509/996] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:36:46 2022] GET /identity/v3/role_assignments?user.id=2b03263937cb4ca7807a24da44d37b0f&scope.project.id=c1eae881e5714bf4bd223809e1a2d920&role.id=7fee85e1d15545a1926ac22137e06585 => generated 610 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:46.548021 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91e554-0034-4dba-9393-8a57ccfa630f None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:46.548555 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91e554-0034-4dba-9393-8a57ccfa630f None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:46.548555 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f91e554-0034-4dba-9393-8a57ccfa630f None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:46.549950 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 488/997] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:46 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:46.555964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8473e-e95b-403a-b016-53e4817f8c86 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:46.556253 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8473e-e95b-403a-b016-53e4817f8c86 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:46.556537 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b8473e-e95b-403a-b016-53e4817f8c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:46.576389 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-03b8473e-e95b-403a-b016-53e4817f8c86 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:46.578142 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-03b8473e-e95b-403a-b016-53e4817f8c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:46.648966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:46.657577 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 510/998] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:46 2022] POST /identity/v3/auth/tokens => generated 1572 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:46.680502 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91e554-0034-4dba-9393-8a57ccfa630f None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:46.725984 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:46.789152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f91e554-0034-4dba-9393-8a57ccfa630f None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:46.825951 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14cf14-a58e-45dd-9186-2d79cf0e1280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:46.826592 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14cf14-a58e-45dd-9186-2d79cf0e1280 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:46.826869 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14cf14-a58e-45dd-9186-2d79cf0e1280 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:46.827172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14cf14-a58e-45dd-9186-2d79cf0e1280 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:46.827799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e14cf14-a58e-45dd-9186-2d79cf0e1280 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:46.829424 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e14cf14-a58e-45dd-9186-2d79cf0e1280 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:46.838756 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-2e14cf14-a58e-45dd-9186-2d79cf0e1280 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:36:46.838756 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:46.838756 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:36:46.838756 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:36:46.838756 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:36:46.838756 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:36:46.838756 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:36:46.838756 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:46.838756 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:36:46.838756 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:46.838756 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:36:46.838756 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:46.838756 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:46.838756 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:46.838756 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:46.838756 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:46.838756 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:46.840630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:46.840630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:46.840630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:46.840630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:46.840630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:46.840630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:46.840630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:36:46.840630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:36:46.840630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:36:46.840630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:36:46.840630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:46.840630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:46.840630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:46.840630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:46.840630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:46.840630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:46.841986 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:46.841986 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:46.841986 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:46.841986 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:46.841986 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:46.841986 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:46.841986 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:46.841986 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:46.841986 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:46.841986 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:46.841986 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:36:46.841986 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:46.841986 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 02 10:36:46.841986 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:46.845485 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 489/999] 10.222.0.115 () {56 vars in 1131 bytes} [Tue Aug 2 10:36:46 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:46.847017 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8473e-e95b-403a-b016-53e4817f8c86 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:46.960129 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b8473e-e95b-403a-b016-53e4817f8c86 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:46.999304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d2d0e-47ac-4f1b-b181-7735efbdf8bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:46.999982 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d2d0e-47ac-4f1b-b181-7735efbdf8bc None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:47.000265 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d2d0e-47ac-4f1b-b181-7735efbdf8bc None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:47.000539 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3d2d0e-47ac-4f1b-b181-7735efbdf8bc None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:47.001221 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3d2d0e-47ac-4f1b-b181-7735efbdf8bc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:47.001526 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3d2d0e-47ac-4f1b-b181-7735efbdf8bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:47.002888 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3d2d0e-47ac-4f1b-b181-7735efbdf8bc None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:47.014796 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 511/1000] 10.222.0.115 () {56 vars in 1127 bytes} [Tue Aug 2 10:36:46 2022] GET /identity/v3/services?name=object-store => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:47.015902 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14cf14-a58e-45dd-9186-2d79cf0e1280 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:47.131170 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14cf14-a58e-45dd-9186-2d79cf0e1280 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:47.132730 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:47.169464 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7ce84-3e90-440f-8327-b9c643aa9e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:47.170237 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7ce84-3e90-440f-8327-b9c643aa9e1e None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:47.170580 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7ce84-3e90-440f-8327-b9c643aa9e1e None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:47.170918 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7ce84-3e90-440f-8327-b9c643aa9e1e None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:47.171685 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b7ce84-3e90-440f-8327-b9c643aa9e1e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:47.172052 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b7ce84-3e90-440f-8327-b9c643aa9e1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:47.173476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b7ce84-3e90-440f-8327-b9c643aa9e1e None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:47.183390 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 490/1001] 10.222.0.115 () {56 vars in 1127 bytes} [Tue Aug 2 10:36:47 2022] GET /identity/v3/services?type=object-store => generated 358 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:47.185212 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d2d0e-47ac-4f1b-b181-7735efbdf8bc None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:47.186270 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1fd71-55ba-4b4d-b364-69178268193b None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:47.186558 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1fd71-55ba-4b4d-b364-69178268193b None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:47.186834 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c1fd71-55ba-4b4d-b364-69178268193b None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:47.188421 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 491/1002] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:47 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:36:47.193446 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b6a1c-a061-4ad1-bc04-953d97e7e815 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:47.193726 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b6a1c-a061-4ad1-bc04-953d97e7e815 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:47.194001 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0b6a1c-a061-4ad1-bc04-953d97e7e815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:47.214397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-2f0b6a1c-a061-4ad1-bc04-953d97e7e815 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:47.216473 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-2f0b6a1c-a061-4ad1-bc04-953d97e7e815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:47.254645 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:47.302227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3d2d0e-47ac-4f1b-b181-7735efbdf8bc None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:47.316673 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 492/1003] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:47 2022] POST /identity/v3/auth/tokens => generated 1572 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:36:47.340956 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b6a1c-a061-4ad1-bc04-953d97e7e815 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:47.343454 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921f48c-828a-4d22-bb71-413b15d1147c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:47.345021 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921f48c-828a-4d22-bb71-413b15d1147c None admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:47.345302 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921f48c-828a-4d22-bb71-413b15d1147c None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:47.345571 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d921f48c-828a-4d22-bb71-413b15d1147c None admin] PATH_INFO: `/v3/endpoints` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:47.346309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921f48c-828a-4d22-bb71-413b15d1147c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:47.347712 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d921f48c-828a-4d22-bb71-413b15d1147c None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:47.414758 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 512/1004] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:36:47 2022] POST /identity/v3/endpoints => generated 327 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:36:47.470265 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:47.477167 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0b6a1c-a061-4ad1-bc04-953d97e7e815 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:47.540238 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a61fe-7b56-4e53-b29c-d621650f8862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:47.540978 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a61fe-7b56-4e53-b29c-d621650f8862 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:47.541255 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a61fe-7b56-4e53-b29c-d621650f8862 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:47.541522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a61fe-7b56-4e53-b29c-d621650f8862 None admin] PATH_INFO: `/v3/services/compute` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:47.542164 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a61fe-7b56-4e53-b29c-d621650f8862 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:47.544767 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a61fe-7b56-4e53-b29c-d621650f8862 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:47.554888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-a25a61fe-7b56-4e53-b29c-d621650f8862 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 10:36:47.554888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:47.554888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:36:47.554888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:36:47.554888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:36:47.554888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:36:47.554888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:36:47.554888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:47.554888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 10:36:47.554888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:47.554888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:36:47.554888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:47.554888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:47.554888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:47.554888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:47.554888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:47.554888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:47.556531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:47.556531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:47.556531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:47.556531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:47.556531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:47.556531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:47.556531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:36:47.556531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:36:47.556531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:36:47.556531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:36:47.556531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:47.556531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:47.556531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:47.556531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:47.556531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:47.556531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:47.558036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:47.558036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:47.558036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:47.558036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:47.558036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:47.558036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:47.558036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:47.558036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:47.558036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:47.558036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:47.558036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:36:47.558036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:47.558036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 02 10:36:47.558036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:47.563279 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 493/1005] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:36:47 2022] GET /identity/v3/services/compute => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:36:47.566479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921f48c-828a-4d22-bb71-413b15d1147c None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:47.716200 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d921f48c-828a-4d22-bb71-413b15d1147c None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:47.737545 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:47.752607 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c20b51-0473-4afe-9e6c-ce20ad7e5177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:47.755543 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c20b51-0473-4afe-9e6c-ce20ad7e5177 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:47.755543 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c20b51-0473-4afe-9e6c-ce20ad7e5177 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:47.755543 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c20b51-0473-4afe-9e6c-ce20ad7e5177 None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:47.755543 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c20b51-0473-4afe-9e6c-ce20ad7e5177 None admin] RBAC: Adding query filter params (name=compute) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:47.755543 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c20b51-0473-4afe-9e6c-ce20ad7e5177 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:47.756199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c20b51-0473-4afe-9e6c-ce20ad7e5177 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:47.765926 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 513/1006] 10.222.0.115 () {56 vars in 1117 bytes} [Tue Aug 2 10:36:47 2022] GET /identity/v3/services?name=compute => generated 125 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:47.767597 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a61fe-7b56-4e53-b29c-d621650f8862 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:47.878043 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a61fe-7b56-4e53-b29c-d621650f8862 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:47.912454 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9845c8-baa1-4670-8ea5-4c66e6adf99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:47.913069 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9845c8-baa1-4670-8ea5-4c66e6adf99d None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:47.913350 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9845c8-baa1-4670-8ea5-4c66e6adf99d None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:47.913629 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9845c8-baa1-4670-8ea5-4c66e6adf99d None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:47.916129 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9845c8-baa1-4670-8ea5-4c66e6adf99d None admin] RBAC: Adding query filter params (type=compute) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:47.916129 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9845c8-baa1-4670-8ea5-4c66e6adf99d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:47.916129 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9845c8-baa1-4670-8ea5-4c66e6adf99d None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:47.922203 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 494/1007] 10.222.0.115 () {56 vars in 1117 bytes} [Tue Aug 2 10:36:47 2022] GET /identity/v3/services?type=compute => generated 354 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:47.927854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c20b51-0473-4afe-9e6c-ce20ad7e5177 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:48.033527 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c20b51-0473-4afe-9e6c-ce20ad7e5177 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:48.069395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e1e8d-d871-48b9-ac0d-049c31b7fb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:48.070029 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e1e8d-d871-48b9-ac0d-049c31b7fb65 None admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:48.070312 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e1e8d-d871-48b9-ac0d-049c31b7fb65 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:48.070596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e1e8d-d871-48b9-ac0d-049c31b7fb65 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:48.071353 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e1e8d-d871-48b9-ac0d-049c31b7fb65 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:48.072748 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928e1e8d-d871-48b9-ac0d-049c31b7fb65 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:48.169171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 514/1008] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:36:47 2022] POST /identity/v3/endpoints => generated 336 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:36:48.582781 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6473afd-5355-4de9-ab28-0be6d3011da0 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:48.583394 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6473afd-5355-4de9-ab28-0be6d3011da0 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:48.583394 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6473afd-5355-4de9-ab28-0be6d3011da0 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:48.584898 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 495/1009] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:48 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:48.590833 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe698d-b9ee-4674-a827-a3f473289866 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:48.591362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe698d-b9ee-4674-a827-a3f473289866 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:48.591458 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffe698d-b9ee-4674-a827-a3f473289866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:48.610045 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-3ffe698d-b9ee-4674-a827-a3f473289866 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:48.611258 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-3ffe698d-b9ee-4674-a827-a3f473289866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:48.651010 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:48.694074 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 515/1010] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:48 2022] POST /identity/v3/auth/tokens => generated 1877 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:48.717614 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6473afd-5355-4de9-ab28-0be6d3011da0 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:48.741606 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:48.827491 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6473afd-5355-4de9-ab28-0be6d3011da0 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:48.861487 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22706ec7-f581-4734-ad08-fa6c187d6de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:48.862121 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22706ec7-f581-4734-ad08-fa6c187d6de1 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:48.862403 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22706ec7-f581-4734-ad08-fa6c187d6de1 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:48.862678 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22706ec7-f581-4734-ad08-fa6c187d6de1 None admin] PATH_INFO: `/v3/services/image` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:48.863326 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22706ec7-f581-4734-ad08-fa6c187d6de1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:48.864730 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22706ec7-f581-4734-ad08-fa6c187d6de1 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:48.874907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-22706ec7-f581-4734-ad08-fa6c187d6de1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 10:36:48.874907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:48.874907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:36:48.874907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:36:48.874907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:36:48.874907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:36:48.874907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:36:48.874907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:48.874907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 10:36:48.874907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:48.874907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:36:48.874907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:48.874907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:48.874907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:48.874907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:48.874907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:48.874907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:48.876114 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:48.876114 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:48.876114 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:48.876114 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:48.876114 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:48.876114 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:48.876114 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:36:48.876114 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:36:48.876114 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:36:48.876114 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:36:48.876114 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:48.876114 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:48.876114 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:48.876114 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:48.876114 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:48.876114 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:48.877015 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:48.877015 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:48.877015 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:48.877015 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:48.877015 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:48.877015 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:48.877015 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:48.877015 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:48.877015 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:48.877015 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:48.877015 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:36:48.877015 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:48.877015 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 10:36:48.877015 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:48.881068 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 496/1011] 10.222.0.115 () {56 vars in 1110 bytes} [Tue Aug 2 10:36:48 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:48.882076 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe698d-b9ee-4674-a827-a3f473289866 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:48.992831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffe698d-b9ee-4674-a827-a3f473289866 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:49.031914 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc44838b-cd47-4691-bde9-bad39efac4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:49.032549 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44838b-cd47-4691-bde9-bad39efac4b7 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:49.032834 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44838b-cd47-4691-bde9-bad39efac4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:49.033110 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc44838b-cd47-4691-bde9-bad39efac4b7 None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:49.033795 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc44838b-cd47-4691-bde9-bad39efac4b7 None admin] RBAC: Adding query filter params (name=image) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:49.034099 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc44838b-cd47-4691-bde9-bad39efac4b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:49.035505 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc44838b-cd47-4691-bde9-bad39efac4b7 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:49.045493 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 516/1012] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:48 2022] GET /identity/v3/services?name=image => generated 123 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:49.047109 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22706ec7-f581-4734-ad08-fa6c187d6de1 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:49.136570 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:49.153445 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22706ec7-f581-4734-ad08-fa6c187d6de1 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:49.189105 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f380c2c-1e82-4d44-8e9e-8488adfdd548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:49.189723 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f380c2c-1e82-4d44-8e9e-8488adfdd548 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:49.190001 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f380c2c-1e82-4d44-8e9e-8488adfdd548 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:49.190273 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f380c2c-1e82-4d44-8e9e-8488adfdd548 None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:49.190947 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f380c2c-1e82-4d44-8e9e-8488adfdd548 None admin] RBAC: Adding query filter params (type=image) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:49.191278 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f380c2c-1e82-4d44-8e9e-8488adfdd548 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:49.192664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f380c2c-1e82-4d44-8e9e-8488adfdd548 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:49.197470 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 497/1013] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:49 2022] GET /identity/v3/services?type=image => generated 123 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:49.494855 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ee64e-57f8-4ee3-b304-086397ae6948 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:49.505993 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ee64e-57f8-4ee3-b304-086397ae6948 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:49.505993 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0ee64e-57f8-4ee3-b304-086397ae6948 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:49.505993 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 517/1014] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:49.506298 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6087f-bda2-4e23-af55-76114b28321b None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:49.506298 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6087f-bda2-4e23-af55-76114b28321b None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:49.506298 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6087f-bda2-4e23-af55-76114b28321b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:49.523190 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-07d6087f-bda2-4e23-af55-76114b28321b None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:49.524369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-07d6087f-bda2-4e23-af55-76114b28321b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:49.594414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 498/1015] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:49 2022] POST /identity/v3/auth/tokens => generated 1877 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:49.617772 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ee64e-57f8-4ee3-b304-086397ae6948 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:49.653295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:49.727195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0ee64e-57f8-4ee3-b304-086397ae6948 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:49.761707 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31362e13-e48d-430a-bc41-d9e68dba48a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:49.763153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31362e13-e48d-430a-bc41-d9e68dba48a0 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:49.763318 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31362e13-e48d-430a-bc41-d9e68dba48a0 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:49.763598 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31362e13-e48d-430a-bc41-d9e68dba48a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:49.764600 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31362e13-e48d-430a-bc41-d9e68dba48a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:49.766032 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31362e13-e48d-430a-bc41-d9e68dba48a0 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:49.767852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 518/1016] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:49 2022] GET /identity/v3/domains/default => generated 200 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:49.773515 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d6087f-bda2-4e23-af55-76114b28321b None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:49.857986 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:49.880201 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d6087f-bda2-4e23-af55-76114b28321b None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:49.916185 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f63f5c-e49e-46a0-8d81-59c2b53bfebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:49.916807 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f63f5c-e49e-46a0-8d81-59c2b53bfebd None admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:49.917083 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f63f5c-e49e-46a0-8d81-59c2b53bfebd None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:49.917350 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f63f5c-e49e-46a0-8d81-59c2b53bfebd None admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:49.918158 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f63f5c-e49e-46a0-8d81-59c2b53bfebd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:49.920066 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f63f5c-e49e-46a0-8d81-59c2b53bfebd None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:50.011507 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 499/1017] 10.222.0.115 () {60 vars in 1146 bytes} [Tue Aug 2 10:36:49 2022] POST /identity/v3/projects => generated 312 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:36:50.360580 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006713d6-414e-431e-ac48-d6454d29c565 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:50.360996 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006713d6-414e-431e-ac48-d6454d29c565 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:50.361120 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006713d6-414e-431e-ac48-d6454d29c565 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:50.363077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 519/1018] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:50 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:50.368328 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b880eb-56af-4cbf-bd9f-7c2f59931345 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:50.368619 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b880eb-56af-4cbf-bd9f-7c2f59931345 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:50.369010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b880eb-56af-4cbf-bd9f-7c2f59931345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:50.388887 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-d4b880eb-56af-4cbf-bd9f-7c2f59931345 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:50.390083 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-d4b880eb-56af-4cbf-bd9f-7c2f59931345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:50.466960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 500/1019] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:50 2022] POST /identity/v3/auth/tokens => generated 1877 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:50.490201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006713d6-414e-431e-ac48-d6454d29c565 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:50.595627 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:50.600700 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006713d6-414e-431e-ac48-d6454d29c565 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:50.637790 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce53e8-ea4a-4aca-855a-18b54a49f745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:50.638439 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce53e8-ea4a-4aca-855a-18b54a49f745 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:50.638676 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce53e8-ea4a-4aca-855a-18b54a49f745 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:50.638895 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ce53e8-ea4a-4aca-855a-18b54a49f745 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:50.649400 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ce53e8-ea4a-4aca-855a-18b54a49f745 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:36:50.655151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-90ce53e8-ea4a-4aca-855a-18b54a49f745 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:36:50.655151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:50.655151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:50.655151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:50.655151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:50.655151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:50.655151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:50.655151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:50.655151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:50.655151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:50.655151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:50.655151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:50.655151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:36:50.655151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:36:50.655151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:36:50.655151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:36:50.656256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:36:50.656256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:36:50.656256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:36:50.656256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:36:50.656256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:50.656256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:50.656256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:50.656256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:50.656256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:50.656256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:50.656256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:50.656256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:50.656256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:50.656256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:50.656256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:50.656256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:50.657195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:50.657195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:50.657195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:50.657195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:50.657195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:36:50.657195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:36:50.657195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:36:50.657195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:36:50.657195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:36:50.657195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:36:50.657195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:36:50.657195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:50.659396 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 520/1020] 10.222.0.115 () {56 vars in 1125 bytes} [Tue Aug 2 10:36:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:50.662269 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b880eb-56af-4cbf-bd9f-7c2f59931345 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:50.728994 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:50.773315 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b880eb-56af-4cbf-bd9f-7c2f59931345 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:50.810751 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df505bd2-0f70-48ad-b11b-c29f1ce1beac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:50.811404 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df505bd2-0f70-48ad-b11b-c29f1ce1beac None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:50.811684 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df505bd2-0f70-48ad-b11b-c29f1ce1beac None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:50.811958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df505bd2-0f70-48ad-b11b-c29f1ce1beac None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:50.812771 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df505bd2-0f70-48ad-b11b-c29f1ce1beac None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:50.813084 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df505bd2-0f70-48ad-b11b-c29f1ce1beac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:50.814453 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df505bd2-0f70-48ad-b11b-c29f1ce1beac None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:50.827945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 501/1021] 10.222.0.115 () {56 vars in 1120 bytes} [Tue Aug 2 10:36:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:50.829412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce53e8-ea4a-4aca-855a-18b54a49f745 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:50.936966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ce53e8-ea4a-4aca-855a-18b54a49f745 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:50.972629 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96764e5-93ad-402d-94ae-3f67fc81877b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:50.973270 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96764e5-93ad-402d-94ae-3f67fc81877b None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:50.973554 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96764e5-93ad-402d-94ae-3f67fc81877b None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:50.973832 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96764e5-93ad-402d-94ae-3f67fc81877b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:50.985369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96764e5-93ad-402d-94ae-3f67fc81877b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:50.986812 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96764e5-93ad-402d-94ae-3f67fc81877b None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:51.001670 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-f96764e5-93ad-402d-94ae-3f67fc81877b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:51.001670 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:51.001670 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:51.001670 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:51.001670 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:51.001670 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:51.001670 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:51.001670 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:51.001670 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:51.001670 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:51.001670 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:51.001670 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:51.001670 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:51.001670 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:51.001670 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:51.002643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:51.002643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:51.002643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:51.002643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:51.002643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:51.002643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:51.002643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:51.002643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:51.002643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:51.002643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:51.002643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:51.002643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:51.002643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:51.002643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:51.002643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:51.002643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:51.002643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:51.008152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:51.008152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:51.008152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:51.008152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:51.008152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 521/1022] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:51.008469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df505bd2-0f70-48ad-b11b-c29f1ce1beac None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:51.119036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df505bd2-0f70-48ad-b11b-c29f1ce1beac None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:51.144259 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:51.157745 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dee76b-3c32-4dcf-8cfd-9702cc8cf3a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:51.158683 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dee76b-3c32-4dcf-8cfd-9702cc8cf3a1 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:51.158683 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dee76b-3c32-4dcf-8cfd-9702cc8cf3a1 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:51.158935 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dee76b-3c32-4dcf-8cfd-9702cc8cf3a1 None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:51.159612 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dee76b-3c32-4dcf-8cfd-9702cc8cf3a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:51.159913 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dee76b-3c32-4dcf-8cfd-9702cc8cf3a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:51.161330 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dee76b-3c32-4dcf-8cfd-9702cc8cf3a1 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:51.174011 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 502/1023] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:51 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:36:51.179476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96764e5-93ad-402d-94ae-3f67fc81877b None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:51.195600 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:51.219140 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d6b38-e230-4678-a549-3df28075bc55 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:51.219489 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d6b38-e230-4678-a549-3df28075bc55 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:51.219698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d6b38-e230-4678-a549-3df28075bc55 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:51.221205 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 503/1024] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:51 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:36:51.226018 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a60b60-bb30-4908-a785-1eef755f0e2c None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:51.226310 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a60b60-bb30-4908-a785-1eef755f0e2c None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:51.226639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a60b60-bb30-4908-a785-1eef755f0e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:51.246556 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-07a60b60-bb30-4908-a785-1eef755f0e2c None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:51.248501 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-07a60b60-bb30-4908-a785-1eef755f0e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:51.295998 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96764e5-93ad-402d-94ae-3f67fc81877b None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:51.321552 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 504/1025] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:51 2022] POST /identity/v3/auth/tokens => generated 1877 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:36:51.332212 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b62dd6-338f-435e-98aa-555bad79144c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:51.332859 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b62dd6-338f-435e-98aa-555bad79144c None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:51.333142 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b62dd6-338f-435e-98aa-555bad79144c None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:51.333502 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b62dd6-338f-435e-98aa-555bad79144c None admin] PATH_INFO: `/v3/users/nova` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:51.342150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b62dd6-338f-435e-98aa-555bad79144c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:51.343630 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b62dd6-338f-435e-98aa-555bad79144c None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:51.345053 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a60b60-bb30-4908-a785-1eef755f0e2c None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:51.356982 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-26b62dd6-338f-435e-98aa-555bad79144c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:36:51.356982 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:51.356982 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:51.356982 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:51.356982 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:51.356982 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:51.356982 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:51.356982 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:51.356982 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:51.356982 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:51.356982 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:51.356982 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:51.356982 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:36:51.356982 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:36:51.356982 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:36:51.356982 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:36:51.358052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:51.358052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:51.358052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:36:51.358052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:51.358052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:36:51.358052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:51.358052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:51.358052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:51.358052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:51.358052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:51.358052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:51.358052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:51.358052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:51.358052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:51.358052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:51.358052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:51.358052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:51.358052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:51.359031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:51.359031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:51.359031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:36:51.359031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:36:51.359031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:36:51.359031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:36:51.359031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:36:51.359031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:36:51.359031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:36:51.359031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:51.359654 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 522/1026] 10.222.0.115 () {56 vars in 1098 bytes} [Tue Aug 2 10:36:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:51.364303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b62dd6-338f-435e-98aa-555bad79144c None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:51.444199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:51.462356 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a60b60-bb30-4908-a785-1eef755f0e2c None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:51.476380 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:51.481373 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b62dd6-338f-435e-98aa-555bad79144c None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:51.498246 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d7505-db7e-450a-a123-5d923b62caf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:51.502101 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d7505-db7e-450a-a123-5d923b62caf9 None admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:51.502101 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d7505-db7e-450a-a123-5d923b62caf9 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:51.502101 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91d7505-db7e-450a-a123-5d923b62caf9 None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:51.502101 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d7505-db7e-450a-a123-5d923b62caf9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:51.502101 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91d7505-db7e-450a-a123-5d923b62caf9 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:51.517838 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d52421-4249-476b-9de9-e75e9139a759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:51.518466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d52421-4249-476b-9de9-e75e9139a759 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:51.518737 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d52421-4249-476b-9de9-e75e9139a759 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:51.519009 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d52421-4249-476b-9de9-e75e9139a759 None admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:51.519824 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d52421-4249-476b-9de9-e75e9139a759 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:51.520130 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d52421-4249-476b-9de9-e75e9139a759 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:51.521542 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d52421-4249-476b-9de9-e75e9139a759 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:51.539830 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 523/1027] 10.222.0.115 () {56 vars in 1138 bytes} [Tue Aug 2 10:36:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:36:51.545423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d52421-4249-476b-9de9-e75e9139a759 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:51.603681 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 505/1028] 10.222.0.115 () {60 vars in 1146 bytes} [Tue Aug 2 10:36:51 2022] POST /identity/v3/services => generated 243 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:36:51.660426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d52421-4249-476b-9de9-e75e9139a759 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:51.691919 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:51.708609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d59b5-10e0-4610-8b05-b8bcf2a34696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:51.709239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d59b5-10e0-4610-8b05-b8bcf2a34696 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:51.709520 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d59b5-10e0-4610-8b05-b8bcf2a34696 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:51.709821 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997d59b5-10e0-4610-8b05-b8bcf2a34696 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:51.789503 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997d59b5-10e0-4610-8b05-b8bcf2a34696 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:51.790962 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997d59b5-10e0-4610-8b05-b8bcf2a34696 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:51.894955 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 524/1029] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:36:51 2022] GET /identity/v3/auth/tokens => generated 1973 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 10:36:51.900192 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d7505-db7e-450a-a123-5d923b62caf9 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:52.005086 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91d7505-db7e-450a-a123-5d923b62caf9 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:52.039035 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb42c6-9b43-4677-a77e-dbf090ebec91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:52.039752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb42c6-9b43-4677-a77e-dbf090ebec91 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:52.039983 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb42c6-9b43-4677-a77e-dbf090ebec91 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:52.040254 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb42c6-9b43-4677-a77e-dbf090ebec91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:52.051072 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb42c6-9b43-4677-a77e-dbf090ebec91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:52.052537 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb42c6-9b43-4677-a77e-dbf090ebec91 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:52.068204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-74fb42c6-9b43-4677-a77e-dbf090ebec91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:52.068204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:52.068204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:52.068204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:52.068204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:52.068204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:52.068204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:52.068204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:52.068204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:52.068204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:52.068204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:52.068204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:52.068204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:52.068204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:52.068204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:52.069227 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:52.069227 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:52.069227 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:52.069227 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:52.069227 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:52.069227 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:52.069227 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:52.069227 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:52.069227 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:52.069227 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:52.069227 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:52.069227 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:52.069227 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:52.069227 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:52.069227 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:52.069227 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:52.069227 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:52.070248 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:52.070248 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:52.070248 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:52.070248 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:52.070248 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 506/1030] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:52.076975 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d59b5-10e0-4610-8b05-b8bcf2a34696 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:52.107263 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:52.129728 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991d040-6707-4c65-a050-07614fc46d09 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:52.130014 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991d040-6707-4c65-a050-07614fc46d09 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:52.130298 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991d040-6707-4c65-a050-07614fc46d09 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:52.131838 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 507/1031] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:52 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:36:52.136646 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747b41a-2efe-472f-a406-4073eea55b65 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:52.136990 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747b41a-2efe-472f-a406-4073eea55b65 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:52.137277 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e747b41a-2efe-472f-a406-4073eea55b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:52.156155 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-e747b41a-2efe-472f-a406-4073eea55b65 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:52.157368 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-e747b41a-2efe-472f-a406-4073eea55b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:52.175961 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:52.189071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997d59b5-10e0-4610-8b05-b8bcf2a34696 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:52.227157 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a582c-ed6f-4c52-9c4b-e31514ed4825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:52.227773 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a582c-ed6f-4c52-9c4b-e31514ed4825 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:52.228053 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a582c-ed6f-4c52-9c4b-e31514ed4825 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:52.228350 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a582c-ed6f-4c52-9c4b-e31514ed4825 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:52.229074 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a582c-ed6f-4c52-9c4b-e31514ed4825 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:52.229340 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a582c-ed6f-4c52-9c4b-e31514ed4825 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:52.231272 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a582c-ed6f-4c52-9c4b-e31514ed4825 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:52.232687 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 508/1032] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:52 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:36:52.247968 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 525/1033] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:52 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:52.249628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747b41a-2efe-472f-a406-4073eea55b65 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:52.256718 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a582c-ed6f-4c52-9c4b-e31514ed4825 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:52.350563 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:52.365335 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e747b41a-2efe-472f-a406-4073eea55b65 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:52.368320 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a582c-ed6f-4c52-9c4b-e31514ed4825 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:52.402478 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80043c3f-8a68-4c95-aa6c-03804b67423d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:52.403368 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80043c3f-8a68-4c95-aa6c-03804b67423d None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:52.403646 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80043c3f-8a68-4c95-aa6c-03804b67423d None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:52.403920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80043c3f-8a68-4c95-aa6c-03804b67423d None admin] PATH_INFO: `/v3/projects/service` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:52.406222 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24bbdf-c0d5-4ffa-9b1f-dc9bcd1d5427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:52.407074 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24bbdf-c0d5-4ffa-9b1f-dc9bcd1d5427 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:52.407484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24bbdf-c0d5-4ffa-9b1f-dc9bcd1d5427 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:52.407665 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d24bbdf-c0d5-4ffa-9b1f-dc9bcd1d5427 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:52.408785 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d24bbdf-c0d5-4ffa-9b1f-dc9bcd1d5427 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:52.410211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d24bbdf-c0d5-4ffa-9b1f-dc9bcd1d5427 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:52.411128 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80043c3f-8a68-4c95-aa6c-03804b67423d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:52.412547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80043c3f-8a68-4c95-aa6c-03804b67423d None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:52.415222 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 526/1034] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:52 2022] GET /identity/v3/domains/default => generated 200 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:36:52.417647 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24bbdf-c0d5-4ffa-9b1f-dc9bcd1d5427 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:52.424686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-80043c3f-8a68-4c95-aa6c-03804b67423d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:52.424686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:52.424686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:52.424686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:52.424686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:52.424686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:52.424686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:52.424686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:52.424686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:52.424686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:52.424686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:52.424686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:52.424686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:36:52.424686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:36:52.424686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:36:52.425639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:36:52.425639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:52.425639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:52.425639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:52.425639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:52.425639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:52.425639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:52.425639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:52.425639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:52.425639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:52.425639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:52.425639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:52.425639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:52.425639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:52.425639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:52.425639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:52.425639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:52.426566 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:36:52.426566 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:36:52.426566 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:36:52.426566 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:36:52.426566 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:36:52.426566 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:36:52.426566 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:52.426566 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:52.426566 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 509/1035] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:36:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:36:52.431569 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80043c3f-8a68-4c95-aa6c-03804b67423d None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:52.449278 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:52.536806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d24bbdf-c0d5-4ffa-9b1f-dc9bcd1d5427 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:52.543977 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80043c3f-8a68-4c95-aa6c-03804b67423d None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:52.571455 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c3ec6f-66d9-49b5-9252-3b33d3586232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:52.572081 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3ec6f-66d9-49b5-9252-3b33d3586232 None admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:52.572360 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3ec6f-66d9-49b5-9252-3b33d3586232 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:52.572632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c3ec6f-66d9-49b5-9252-3b33d3586232 None admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:52.573317 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c3ec6f-66d9-49b5-9252-3b33d3586232 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:52.575347 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c3ec6f-66d9-49b5-9252-3b33d3586232 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:52.581279 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1573bb1-4946-44b2-8350-1a2605587608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:52.582068 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1573bb1-4946-44b2-8350-1a2605587608 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:52.582489 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1573bb1-4946-44b2-8350-1a2605587608 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:52.582934 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1573bb1-4946-44b2-8350-1a2605587608 None admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:52.583852 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1573bb1-4946-44b2-8350-1a2605587608 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:52.584319 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1573bb1-4946-44b2-8350-1a2605587608 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:52.585874 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1573bb1-4946-44b2-8350-1a2605587608 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:52.599643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 510/1036] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:36:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:36:52.605995 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1573bb1-4946-44b2-8350-1a2605587608 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:52.658972 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 527/1037] 10.222.0.115 () {60 vars in 1137 bytes} [Tue Aug 2 10:36:52 2022] POST /identity/v3/users => generated 286 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:36:52.691980 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:52.725179 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1573bb1-4946-44b2-8350-1a2605587608 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:52.766351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a52d9c6-2e6a-468d-b6ad-b49ee2cbbccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:52.766978 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52d9c6-2e6a-468d-b6ad-b49ee2cbbccd None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:52.767292 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52d9c6-2e6a-468d-b6ad-b49ee2cbbccd None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:52.767563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a52d9c6-2e6a-468d-b6ad-b49ee2cbbccd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:52.768362 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a52d9c6-2e6a-468d-b6ad-b49ee2cbbccd None admin] RBAC: Adding query filter params (role.id=7fee85e1d15545a1926ac22137e06585, scope.project.id=c1eae881e5714bf4bd223809e1a2d920, user.id=d0478bd540f446ec810e2091c46ac49c) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:52.768671 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a52d9c6-2e6a-468d-b6ad-b49ee2cbbccd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:52.770136 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a52d9c6-2e6a-468d-b6ad-b49ee2cbbccd None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:52.783082 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 511/1038] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:36:52 2022] GET /identity/v3/role_assignments?user.id=d0478bd540f446ec810e2091c46ac49c&scope.project.id=c1eae881e5714bf4bd223809e1a2d920&role.id=7fee85e1d15545a1926ac22137e06585 => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:36:53.661295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bf7cb-5fd7-40df-9210-01b57b1d0708 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:53.661859 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bf7cb-5fd7-40df-9210-01b57b1d0708 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:53.661859 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935bf7cb-5fd7-40df-9210-01b57b1d0708 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:53.663371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 528/1039] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:53 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:53.669111 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f30d2-2ae5-452a-9f70-bcc0ba4026f2 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:53.669590 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f30d2-2ae5-452a-9f70-bcc0ba4026f2 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:53.669861 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f30d2-2ae5-452a-9f70-bcc0ba4026f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:53.688083 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-fc9f30d2-2ae5-452a-9f70-bcc0ba4026f2 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:53.689268 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-fc9f30d2-2ae5-452a-9f70-bcc0ba4026f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:53.757051 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 512/1040] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:53 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:53.782004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935bf7cb-5fd7-40df-9210-01b57b1d0708 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:53.794166 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:53.894406 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935bf7cb-5fd7-40df-9210-01b57b1d0708 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:53.932972 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4fcd61-12ca-47b1-bfef-c49ef71ff9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:53.933665 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4fcd61-12ca-47b1-bfef-c49ef71ff9f8 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:53.933944 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4fcd61-12ca-47b1-bfef-c49ef71ff9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:53.934217 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4fcd61-12ca-47b1-bfef-c49ef71ff9f8 None admin] PATH_INFO: `/v3/services/image` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:53.934848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4fcd61-12ca-47b1-bfef-c49ef71ff9f8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:53.936251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4fcd61-12ca-47b1-bfef-c49ef71ff9f8 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:53.945336 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-bb4fcd61-12ca-47b1-bfef-c49ef71ff9f8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 10:36:53.945336 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:53.945336 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:36:53.945336 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:36:53.945336 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:36:53.945336 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:36:53.945336 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:36:53.945336 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:53.945336 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 10:36:53.945336 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:53.945336 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:36:53.945336 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:53.945336 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:53.945336 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:53.945336 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:53.945336 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:53.945336 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:53.946411 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:53.946411 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:53.946411 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:53.946411 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:53.946411 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:53.946411 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:53.946411 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:36:53.946411 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:36:53.946411 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:36:53.946411 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:36:53.946411 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:53.946411 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:53.946411 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:53.946411 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:53.946411 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:53.946411 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:53.947678 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:53.947678 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:53.947678 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:53.947678 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:53.947678 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:53.947678 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:53.947678 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:53.947678 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:53.947678 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:53.947678 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:53.947678 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:36:53.947678 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:53.947678 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 10:36:53.947678 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:53.951129 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 529/1041] 10.222.0.115 () {56 vars in 1110 bytes} [Tue Aug 2 10:36:53 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:53.953692 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f30d2-2ae5-452a-9f70-bcc0ba4026f2 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:54.020424 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:54.058536 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f30d2-2ae5-452a-9f70-bcc0ba4026f2 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:54.098685 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ba902-873b-48d3-bf6d-08f4f11a0e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:54.099329 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ba902-873b-48d3-bf6d-08f4f11a0e6b None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:54.099606 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ba902-873b-48d3-bf6d-08f4f11a0e6b None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:54.099874 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9ba902-873b-48d3-bf6d-08f4f11a0e6b None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:54.100548 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9ba902-873b-48d3-bf6d-08f4f11a0e6b None admin] RBAC: Adding query filter params (name=image) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:54.100846 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9ba902-873b-48d3-bf6d-08f4f11a0e6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:54.102197 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9ba902-873b-48d3-bf6d-08f4f11a0e6b None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:54.107513 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 513/1042] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:53 2022] GET /identity/v3/services?name=image => generated 123 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:54.113660 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4fcd61-12ca-47b1-bfef-c49ef71ff9f8 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:54.192495 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:54.224092 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4fcd61-12ca-47b1-bfef-c49ef71ff9f8 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:54.260305 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c37c20-4aa7-4f38-bed9-d8013c6397ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:54.260940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c37c20-4aa7-4f38-bed9-d8013c6397ad None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:54.261211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c37c20-4aa7-4f38-bed9-d8013c6397ad None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:54.261529 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c37c20-4aa7-4f38-bed9-d8013c6397ad None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:54.262317 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c37c20-4aa7-4f38-bed9-d8013c6397ad None admin] RBAC: Adding query filter params (type=image) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:54.262647 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c37c20-4aa7-4f38-bed9-d8013c6397ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:54.264036 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c37c20-4aa7-4f38-bed9-d8013c6397ad None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:54.269850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 530/1043] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:54 2022] GET /identity/v3/services?type=image => generated 352 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:36:54.275548 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ba902-873b-48d3-bf6d-08f4f11a0e6b None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:54.377353 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9ba902-873b-48d3-bf6d-08f4f11a0e6b None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:54.407312 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:54.412739 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a116bd-ca34-4409-97b7-6230a7a2d405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:54.413358 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a116bd-ca34-4409-97b7-6230a7a2d405 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:54.413748 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a116bd-ca34-4409-97b7-6230a7a2d405 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:54.414025 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a116bd-ca34-4409-97b7-6230a7a2d405 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:54.414774 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a116bd-ca34-4409-97b7-6230a7a2d405 None admin] RBAC: Adding query filter params (interface=public, service_id=82f5d82804384924a8475afa45f88c5c, region_id=RegionOne) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:54.415102 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a116bd-ca34-4409-97b7-6230a7a2d405 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:54.416463 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a116bd-ca34-4409-97b7-6230a7a2d405 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:54.421922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 514/1044] 10.222.0.115 () {56 vars in 1256 bytes} [Tue Aug 2 10:36:54 2022] GET /identity/v3/endpoints?service_id=82f5d82804384924a8475afa45f88c5c&interface=public®ion_id=RegionOne => generated 195 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:54.427401 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c37c20-4aa7-4f38-bed9-d8013c6397ad None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:54.536958 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c37c20-4aa7-4f38-bed9-d8013c6397ad None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:54.571647 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04671bdd-ce55-4c5b-b9d5-2441ff44df24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:54.572309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04671bdd-ce55-4c5b-b9d5-2441ff44df24 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:54.572593 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04671bdd-ce55-4c5b-b9d5-2441ff44df24 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:54.572868 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04671bdd-ce55-4c5b-b9d5-2441ff44df24 None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:54.573524 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04671bdd-ce55-4c5b-b9d5-2441ff44df24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:54.575861 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04671bdd-ce55-4c5b-b9d5-2441ff44df24 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:54.577864 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:54.587409 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 531/1045] 10.222.0.115 () {56 vars in 1092 bytes} [Tue Aug 2 10:36:54 2022] GET /identity/v3/services => generated 1966 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:36:54.804702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f2131-10bc-4573-a8b2-1fd5390978c8 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:54.805231 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f2131-10bc-4573-a8b2-1fd5390978c8 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:54.805231 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f2131-10bc-4573-a8b2-1fd5390978c8 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:54.806683 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 515/1046] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:54 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:54.812802 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806e630-b2d0-4dd2-94df-564ec71f010c None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:54.813229 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806e630-b2d0-4dd2-94df-564ec71f010c None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:54.813422 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806e630-b2d0-4dd2-94df-564ec71f010c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:54.822578 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efdf01-c0f5-40f7-a8aa-f741b6f5f898 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:54.822874 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efdf01-c0f5-40f7-a8aa-f741b6f5f898 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:54.823159 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efdf01-c0f5-40f7-a8aa-f741b6f5f898 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:54.824990 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 516/1047] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:54 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:54.830612 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643d42e-7047-4d39-8c89-8019b5822514 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:54.830892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643d42e-7047-4d39-8c89-8019b5822514 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:54.831191 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a643d42e-7047-4d39-8c89-8019b5822514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:54.832362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-2806e630-b2d0-4dd2-94df-564ec71f010c None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:54.833548 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-2806e630-b2d0-4dd2-94df-564ec71f010c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:54.849904 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-a643d42e-7047-4d39-8c89-8019b5822514 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:54.851106 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-a643d42e-7047-4d39-8c89-8019b5822514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:54.909121 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 532/1048] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:54 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:54.929389 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 517/1049] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:54 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:36:54.932829 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2806e630-b2d0-4dd2-94df-564ec71f010c None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:54.953952 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643d42e-7047-4d39-8c89-8019b5822514 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:55.041081 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2806e630-b2d0-4dd2-94df-564ec71f010c None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:55.064253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a643d42e-7047-4d39-8c89-8019b5822514 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:55.076314 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:55.082886 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60cb1e-d1bd-4b27-a814-b44b9f3d72f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:55.083563 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60cb1e-d1bd-4b27-a814-b44b9f3d72f4 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:55.083848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60cb1e-d1bd-4b27-a814-b44b9f3d72f4 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:55.084122 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad60cb1e-d1bd-4b27-a814-b44b9f3d72f4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:55.094554 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:55.095026 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad60cb1e-d1bd-4b27-a814-b44b9f3d72f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:36:55.100150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-ad60cb1e-d1bd-4b27-a814-b44b9f3d72f4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:36:55.100150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:55.100150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:55.100150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:55.100150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:55.100150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:55.100150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:55.100150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:55.100150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:55.100150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:55.100150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:55.100150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:55.100150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:36:55.100150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:36:55.100150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:36:55.100150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:36:55.101324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:36:55.101324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:36:55.101324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:36:55.101324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:36:55.101324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:55.101324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:55.101324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:55.101324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:55.101324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:55.101324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:55.101324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:55.101324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:55.101324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:55.101324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:55.101324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:55.101324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:55.102196 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:55.102196 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:55.102196 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:55.102196 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:55.102196 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:36:55.102196 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:36:55.102196 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:36:55.102196 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:36:55.102196 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:36:55.102196 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:36:55.102196 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:36:55.102196 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:55.106304 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169358de-5e55-4f39-a51e-fafeadc74211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:55.106627 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 533/1050] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:36:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:55.107163 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169358de-5e55-4f39-a51e-fafeadc74211 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:55.107343 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169358de-5e55-4f39-a51e-fafeadc74211 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:55.107610 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169358de-5e55-4f39-a51e-fafeadc74211 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:55.107993 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60cb1e-d1bd-4b27-a814-b44b9f3d72f4 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:55.118812 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169358de-5e55-4f39-a51e-fafeadc74211 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:36:55.124356 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-169358de-5e55-4f39-a51e-fafeadc74211 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:36:55.124356 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:55.124356 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:55.124356 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:55.124356 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:55.124356 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:55.124356 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:55.124356 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:55.124356 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:55.124356 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:55.124356 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:55.124356 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:55.124356 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:36:55.124356 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:36:55.124356 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:36:55.124356 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:36:55.125341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:36:55.125341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:36:55.125341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:36:55.125341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:36:55.125341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:55.125341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:55.125341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:55.125341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:55.125341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:55.125341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:55.125341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:55.125341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:55.125341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:55.125341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:55.125341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:55.125341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:55.126203 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:55.126203 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:55.126203 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:55.126203 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:55.126203 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:36:55.126203 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:36:55.126203 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:36:55.126203 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:36:55.126203 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:36:55.126203 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:36:55.126203 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:36:55.126203 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:55.126203 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 518/1051] 10.222.0.115 () {56 vars in 1125 bytes} [Tue Aug 2 10:36:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:36:55.131578 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169358de-5e55-4f39-a51e-fafeadc74211 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:55.220532 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad60cb1e-d1bd-4b27-a814-b44b9f3d72f4 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:55.243935 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169358de-5e55-4f39-a51e-fafeadc74211 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:55.256292 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9c4b7-b329-48e3-b24a-f505d5aa0a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:55.256924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9c4b7-b329-48e3-b24a-f505d5aa0a7f None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:55.257209 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9c4b7-b329-48e3-b24a-f505d5aa0a7f None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:55.257488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd9c4b7-b329-48e3-b24a-f505d5aa0a7f None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:55.258218 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd9c4b7-b329-48e3-b24a-f505d5aa0a7f None admin] RBAC: Adding query filter params (name=admin) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:55.258518 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd9c4b7-b329-48e3-b24a-f505d5aa0a7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:55.259910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd9c4b7-b329-48e3-b24a-f505d5aa0a7f None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:55.269137 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 534/1052] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:36:55 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:36:55.274924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9c4b7-b329-48e3-b24a-f505d5aa0a7f None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:55.292897 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3ec3d-ff44-4ea2-a121-6d8ee51bf57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:55.293847 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3ec3d-ff44-4ea2-a121-6d8ee51bf57c None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:55.294059 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3ec3d-ff44-4ea2-a121-6d8ee51bf57c None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:55.294331 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce3ec3d-ff44-4ea2-a121-6d8ee51bf57c None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:55.295237 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce3ec3d-ff44-4ea2-a121-6d8ee51bf57c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:55.295846 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce3ec3d-ff44-4ea2-a121-6d8ee51bf57c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:55.297213 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce3ec3d-ff44-4ea2-a121-6d8ee51bf57c None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:55.313862 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 519/1053] 10.222.0.115 () {56 vars in 1120 bytes} [Tue Aug 2 10:36:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:36:55.315094 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3ec3d-ff44-4ea2-a121-6d8ee51bf57c None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:55.330704 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:55.377833 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:55.384735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd9c4b7-b329-48e3-b24a-f505d5aa0a7f None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:55.424866 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df520d-dee3-4477-ae68-07fe49888e45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:55.425506 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df520d-dee3-4477-ae68-07fe49888e45 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:55.425790 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df520d-dee3-4477-ae68-07fe49888e45 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:55.426065 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43df520d-dee3-4477-ae68-07fe49888e45 None admin] PATH_INFO: `/v3/users/78e7cd7f62674c558e5c42128ee04634` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:55.439513 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce3ec3d-ff44-4ea2-a121-6d8ee51bf57c None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:55.440137 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df520d-dee3-4477-ae68-07fe49888e45 None admin] RBAC: Authorizing `identity:get_user(user_id=78e7cd7f62674c558e5c42128ee04634)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:55.441575 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43df520d-dee3-4477-ae68-07fe49888e45 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:55.447943 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 535/1054] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:36:55 2022] GET /identity/v3/users/78e7cd7f62674c558e5c42128ee04634 => generated 286 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:36:55.449488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df520d-dee3-4477-ae68-07fe49888e45 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:55.483008 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76e923-d0bf-4a63-85b6-1d2966f0c5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:55.483720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76e923-d0bf-4a63-85b6-1d2966f0c5b7 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:55.484009 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76e923-d0bf-4a63-85b6-1d2966f0c5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:55.484281 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f76e923-d0bf-4a63-85b6-1d2966f0c5b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:55.496731 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76e923-d0bf-4a63-85b6-1d2966f0c5b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:55.498343 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f76e923-d0bf-4a63-85b6-1d2966f0c5b7 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:55.514752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-9f76e923-d0bf-4a63-85b6-1d2966f0c5b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:55.514752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:55.514752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:55.514752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:55.514752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:55.514752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:55.514752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:55.514752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:55.514752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:55.514752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:55.514752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:55.514752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:55.514752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:55.514752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:55.514752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:55.520930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:55.520930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:55.520930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:55.520930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:55.520930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:55.520930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:55.520930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:55.520930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:55.520930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:55.520930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:55.520930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:55.520930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:55.520930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:55.520930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:55.520930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:55.520930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:55.520930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:55.522125 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:55.522125 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:55.522125 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:55.522125 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:55.522125 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 520/1055] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:55.522125 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76e923-d0bf-4a63-85b6-1d2966f0c5b7 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:55.558485 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43df520d-dee3-4477-ae68-07fe49888e45 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:55.569759 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:55.595077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a2608-1301-4784-9c70-21641a83665e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:55.595787 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a2608-1301-4784-9c70-21641a83665e None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:55.596137 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a2608-1301-4784-9c70-21641a83665e None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:55.596416 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752a2608-1301-4784-9c70-21641a83665e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:55.627339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:55.630996 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f76e923-d0bf-4a63-85b6-1d2966f0c5b7 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:55.667181 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a2608-1301-4784-9c70-21641a83665e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:55.668043 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058d10c-1a8b-48c6-9b37-07370848d632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:55.668842 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752a2608-1301-4784-9c70-21641a83665e None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:55.668961 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058d10c-1a8b-48c6-9b37-07370848d632 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:55.669256 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058d10c-1a8b-48c6-9b37-07370848d632 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:55.669590 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e058d10c-1a8b-48c6-9b37-07370848d632 None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:55.670342 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e058d10c-1a8b-48c6-9b37-07370848d632 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:55.670801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e058d10c-1a8b-48c6-9b37-07370848d632 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:55.672250 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e058d10c-1a8b-48c6-9b37-07370848d632 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:55.685608 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 521/1056] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:55 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:36:55.691011 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058d10c-1a8b-48c6-9b37-07370848d632 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:55.785201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 536/1057] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:36:55 2022] GET /identity/v3/auth/tokens => generated 1973 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 10:36:55.786252 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a2608-1301-4784-9c70-21641a83665e None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:55.799366 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:55.801090 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e058d10c-1a8b-48c6-9b37-07370848d632 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:55.836565 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf18dfec-d8b0-4d93-91f7-3baa678b3caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:55.837185 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf18dfec-d8b0-4d93-91f7-3baa678b3caa None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:55.837400 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf18dfec-d8b0-4d93-91f7-3baa678b3caa None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:55.837626 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf18dfec-d8b0-4d93-91f7-3baa678b3caa None admin] PATH_INFO: `/v3/users/nova` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:55.846747 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf18dfec-d8b0-4d93-91f7-3baa678b3caa None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:55.848433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf18dfec-d8b0-4d93-91f7-3baa678b3caa None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:55.862327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-cf18dfec-d8b0-4d93-91f7-3baa678b3caa None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:36:55.862327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:55.862327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:55.862327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:55.862327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:55.862327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:55.862327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:55.862327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:55.862327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:55.862327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:55.862327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:55.862327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:55.862327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:36:55.862327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:36:55.862327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:36:55.862327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:36:55.863799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:55.863799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:55.863799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:36:55.863799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:55.863799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:36:55.863799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:36:55.863799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:55.863799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:55.863799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:55.863799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:55.863799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:55.863799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:55.863799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:55.863799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:55.863799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:55.863799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:55.863799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:55.863799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:55.865016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:55.865016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:55.865016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:36:55.865016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:36:55.865016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:36:55.865016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:36:55.865016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:36:55.865016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:36:55.865016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:36:55.865016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:55.865016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 522/1058] 10.222.0.115 () {56 vars in 1098 bytes} [Tue Aug 2 10:36:55 2022] GET /identity/v3/users/nova => generated 82 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:55.869049 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf18dfec-d8b0-4d93-91f7-3baa678b3caa None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:55.892014 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:55.898909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752a2608-1301-4784-9c70-21641a83665e None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:55.938310 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74654dd0-5299-4827-a93c-170bf276b811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:55.939183 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74654dd0-5299-4827-a93c-170bf276b811 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:55.939466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74654dd0-5299-4827-a93c-170bf276b811 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:55.939739 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74654dd0-5299-4827-a93c-170bf276b811 None admin] PATH_INFO: `/v3/projects/fcb9e5d02067447099445d3a4c0e470b` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:55.940898 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74654dd0-5299-4827-a93c-170bf276b811 None admin] RBAC: Authorizing `identity:get_project(project_id=fcb9e5d02067447099445d3a4c0e470b)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:55.942447 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74654dd0-5299-4827-a93c-170bf276b811 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:55.944146 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 537/1059] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:36:55 2022] GET /identity/v3/projects/fcb9e5d02067447099445d3a4c0e470b => generated 312 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:55.950632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74654dd0-5299-4827-a93c-170bf276b811 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:55.986334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf18dfec-d8b0-4d93-91f7-3baa678b3caa None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:56.021967 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54180ae-46e6-45c7-8e3d-3f688002df38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:56.022581 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54180ae-46e6-45c7-8e3d-3f688002df38 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:56.022859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54180ae-46e6-45c7-8e3d-3f688002df38 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:56.023157 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54180ae-46e6-45c7-8e3d-3f688002df38 None admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:56.023953 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54180ae-46e6-45c7-8e3d-3f688002df38 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:56.024250 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54180ae-46e6-45c7-8e3d-3f688002df38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:56.025719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54180ae-46e6-45c7-8e3d-3f688002df38 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:56.045099 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 523/1060] 10.222.0.115 () {56 vars in 1138 bytes} [Tue Aug 2 10:36:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:36:56.050713 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54180ae-46e6-45c7-8e3d-3f688002df38 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:56.054933 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:56.080756 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74654dd0-5299-4827-a93c-170bf276b811 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:56.122333 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d35c5-3af9-40b2-ab5b-cff91f33b829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:56.122865 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d35c5-3af9-40b2-ab5b-cff91f33b829 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:56.123172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d35c5-3af9-40b2-ab5b-cff91f33b829 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:56.123447 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745d35c5-3af9-40b2-ab5b-cff91f33b829 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:56.124230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d35c5-3af9-40b2-ab5b-cff91f33b829 None admin] RBAC: Adding query filter params (role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac, scope.project.id=fcb9e5d02067447099445d3a4c0e470b, user.id=78e7cd7f62674c558e5c42128ee04634) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:56.124536 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d35c5-3af9-40b2-ab5b-cff91f33b829 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:56.125954 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745d35c5-3af9-40b2-ab5b-cff91f33b829 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:56.131698 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 538/1061] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:36:55 2022] GET /identity/v3/role_assignments?user.id=78e7cd7f62674c558e5c42128ee04634&scope.project.id=fcb9e5d02067447099445d3a4c0e470b&role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:36:56.144355 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:56.168897 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54180ae-46e6-45c7-8e3d-3f688002df38 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:56.209506 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e0b75-1117-4817-ba72-1acd6b02fa74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:56.210166 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e0b75-1117-4817-ba72-1acd6b02fa74 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:56.210447 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e0b75-1117-4817-ba72-1acd6b02fa74 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:56.210723 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425e0b75-1117-4817-ba72-1acd6b02fa74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:56.223092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e0b75-1117-4817-ba72-1acd6b02fa74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:56.224435 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425e0b75-1117-4817-ba72-1acd6b02fa74 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:56.242761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-425e0b75-1117-4817-ba72-1acd6b02fa74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:56.242761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:56.242761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:56.242761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:56.242761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:56.242761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:56.242761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:56.242761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:56.242761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:56.242761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:56.242761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:56.242761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:56.242761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:56.242761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:56.242761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:56.244012 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:56.244012 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:56.244012 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:56.244012 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:56.244012 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:56.244012 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:56.244012 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:56.244012 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:56.244012 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:56.244012 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:56.244012 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:56.244012 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:56.244012 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:56.244012 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:56.244012 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:56.244012 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:56.244012 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:56.251011 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:56.251011 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:56.251011 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:56.251011 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:56.251011 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 524/1062] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:36:56.252678 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d35c5-3af9-40b2-ab5b-cff91f33b829 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:56.373636 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745d35c5-3af9-40b2-ab5b-cff91f33b829 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:56.414711 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9fe36d-e132-4d2e-aa10-1c5a4552e1e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:56.415368 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9fe36d-e132-4d2e-aa10-1c5a4552e1e1 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:56.415648 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9fe36d-e132-4d2e-aa10-1c5a4552e1e1 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:56.415922 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9fe36d-e132-4d2e-aa10-1c5a4552e1e1 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:56.416626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9fe36d-e132-4d2e-aa10-1c5a4552e1e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:56.416931 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9fe36d-e132-4d2e-aa10-1c5a4552e1e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:56.418997 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9fe36d-e132-4d2e-aa10-1c5a4552e1e1 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:56.435091 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 539/1063] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:56 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:56.438103 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e0b75-1117-4817-ba72-1acd6b02fa74 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:56.548558 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425e0b75-1117-4817-ba72-1acd6b02fa74 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:56.577396 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:56.584014 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a35a0e7-295c-46da-bac2-2bd7c3765e71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:56.594845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35a0e7-295c-46da-bac2-2bd7c3765e71 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:56.594845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35a0e7-295c-46da-bac2-2bd7c3765e71 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:56.594845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a35a0e7-295c-46da-bac2-2bd7c3765e71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:56.594845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35a0e7-295c-46da-bac2-2bd7c3765e71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:56.595303 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a35a0e7-295c-46da-bac2-2bd7c3765e71 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:56.608119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-5a35a0e7-295c-46da-bac2-2bd7c3765e71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:56.608119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:56.608119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:56.608119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:56.608119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:56.608119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:56.608119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:56.608119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:56.608119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:56.608119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:56.608119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:56.608119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:56.608119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:36:56.608119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:36:56.608119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:36:56.609296 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:36:56.609296 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:56.609296 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:56.609296 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:56.609296 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:56.609296 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:56.609296 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:56.609296 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:56.609296 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:56.609296 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:56.609296 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:56.609296 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:56.609296 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:56.609296 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:56.609296 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:56.609296 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:56.609296 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:56.610482 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:36:56.610482 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:36:56.610482 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:36:56.610482 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:36:56.610482 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:36:56.610482 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:36:56.610482 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:36:56.610482 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:56.612068 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 525/1064] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:36:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:56.618060 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9fe36d-e132-4d2e-aa10-1c5a4552e1e1 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:56.621359 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:56.642184 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a4841a-c0c3-4d0e-9b2c-cdb3ffd5ed7b None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:56.642369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a4841a-c0c3-4d0e-9b2c-cdb3ffd5ed7b None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:56.642640 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a4841a-c0c3-4d0e-9b2c-cdb3ffd5ed7b None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:56.644240 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 526/1065] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:56.649190 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c04c8-b798-4d38-a565-5af25e6ee9c0 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:56.649471 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c04c8-b798-4d38-a565-5af25e6ee9c0 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:56.649741 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3c04c8-b798-4d38-a565-5af25e6ee9c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:56.669904 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-6d3c04c8-b798-4d38-a565-5af25e6ee9c0 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:56.669904 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-6d3c04c8-b798-4d38-a565-5af25e6ee9c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:56.736451 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9fe36d-e132-4d2e-aa10-1c5a4552e1e1 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:56.753711 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 527/1066] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:56 2022] POST /identity/v3/auth/tokens => generated 1973 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:36:56.773902 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8588889-c930-4371-b025-fe935c25b4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:56.774527 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8588889-c930-4371-b025-fe935c25b4bf None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:56.774816 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8588889-c930-4371-b025-fe935c25b4bf None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:56.775117 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8588889-c930-4371-b025-fe935c25b4bf None admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:56.775732 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3c04c8-b798-4d38-a565-5af25e6ee9c0 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:56.775905 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8588889-c930-4371-b025-fe935c25b4bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:56.776180 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8588889-c930-4371-b025-fe935c25b4bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:56.777675 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8588889-c930-4371-b025-fe935c25b4bf None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:56.792504 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 540/1067] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:36:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:56.798756 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8588889-c930-4371-b025-fe935c25b4bf None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:56.868156 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:56.892685 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3c04c8-b798-4d38-a565-5af25e6ee9c0 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:56.914520 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8588889-c930-4371-b025-fe935c25b4bf None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:56.915872 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:56.931983 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b441e98-3778-4af8-abbb-a7fe5e5cef0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:56.932763 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b441e98-3778-4af8-abbb-a7fe5e5cef0e None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:56.933050 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b441e98-3778-4af8-abbb-a7fe5e5cef0e None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:56.933977 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b441e98-3778-4af8-abbb-a7fe5e5cef0e None admin] PATH_INFO: `/v3/services/image` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:56.934633 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b441e98-3778-4af8-abbb-a7fe5e5cef0e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:56.936033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b441e98-3778-4af8-abbb-a7fe5e5cef0e None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:56.944920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-1b441e98-3778-4af8-abbb-a7fe5e5cef0e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 10:36:56.944920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:56.944920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:36:56.944920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:36:56.944920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:36:56.944920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:36:56.944920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:36:56.944920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:56.944920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 10:36:56.944920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:56.944920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:36:56.944920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:56.944920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:56.944920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:56.944920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:56.944920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:56.944920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:56.945967 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:56.945967 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:56.945967 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:56.945967 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:56.945967 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:56.945967 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:56.945967 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:36:56.945967 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:36:56.945967 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:36:56.945967 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:36:56.945967 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:56.945967 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:56.945967 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:56.945967 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:56.945967 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:56.945967 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:56.947402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:56.947402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:56.947402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:56.947402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:56.947402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:56.947402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:56.947402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:56.947402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:56.947402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:56.947402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:56.947402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:36:56.947402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:36:56.947402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 02 10:36:56.947402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:56.947402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 528/1068] 10.222.0.115 () {56 vars in 1110 bytes} [Tue Aug 2 10:36:56 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:56.952643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b441e98-3778-4af8-abbb-a7fe5e5cef0e None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:56.958221 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1166a1c-58bc-43e1-b73b-30d34cf1c81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:56.958899 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1166a1c-58bc-43e1-b73b-30d34cf1c81e None admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:56.959214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1166a1c-58bc-43e1-b73b-30d34cf1c81e None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:56.959545 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1166a1c-58bc-43e1-b73b-30d34cf1c81e None admin] PATH_INFO: `/v3/projects/c1eae881e5714bf4bd223809e1a2d920/users/d0478bd540f446ec810e2091c46ac49c/roles/7fee85e1d15545a1926ac22137e06585` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:56.961762 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1166a1c-58bc-43e1-b73b-30d34cf1c81e None admin] RBAC: Authorizing `identity:create_grant(project_id=c1eae881e5714bf4bd223809e1a2d920, user_id=d0478bd540f446ec810e2091c46ac49c, role_id=7fee85e1d15545a1926ac22137e06585)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:56.963238 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1166a1c-58bc-43e1-b73b-30d34cf1c81e None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:57.055751 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b441e98-3778-4af8-abbb-a7fe5e5cef0e None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:57.060689 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 541/1069] 10.222.0.115 () {58 vars in 1444 bytes} [Tue Aug 2 10:36:56 2022] PUT /identity/v3/projects/c1eae881e5714bf4bd223809e1a2d920/users/d0478bd540f446ec810e2091c46ac49c/roles/7fee85e1d15545a1926ac22137e06585 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:36:57.096442 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc31a3f4-c233-4205-8f14-cb35f6e9af1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:57.097089 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31a3f4-c233-4205-8f14-cb35f6e9af1f None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:57.097359 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31a3f4-c233-4205-8f14-cb35f6e9af1f None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:57.097633 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc31a3f4-c233-4205-8f14-cb35f6e9af1f None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:57.098335 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc31a3f4-c233-4205-8f14-cb35f6e9af1f None admin] RBAC: Adding query filter params (name=image) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:57.098641 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc31a3f4-c233-4205-8f14-cb35f6e9af1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:57.100086 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc31a3f4-c233-4205-8f14-cb35f6e9af1f None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:57.111095 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 529/1070] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:56 2022] GET /identity/v3/services?name=image => generated 123 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:57.113092 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1166a1c-58bc-43e1-b73b-30d34cf1c81e None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:57.240611 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1166a1c-58bc-43e1-b73b-30d34cf1c81e None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:57.267711 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:57.280503 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9426e1-a666-411f-9d65-0632523edc05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:57.281134 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9426e1-a666-411f-9d65-0632523edc05 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:57.281410 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9426e1-a666-411f-9d65-0632523edc05 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:57.282641 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9426e1-a666-411f-9d65-0632523edc05 None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:57.282641 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9426e1-a666-411f-9d65-0632523edc05 None admin] RBAC: Adding query filter params (type=image) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:57.282641 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9426e1-a666-411f-9d65-0632523edc05 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:57.284052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9426e1-a666-411f-9d65-0632523edc05 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:57.289635 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 542/1071] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:57 2022] GET /identity/v3/services?type=image => generated 352 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:57.295732 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc31a3f4-c233-4205-8f14-cb35f6e9af1f None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:57.350276 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:57.403885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc31a3f4-c233-4205-8f14-cb35f6e9af1f None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:57.444239 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a83c6f-ca08-4859-b4d4-651498fdd061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:57.444854 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a83c6f-ca08-4859-b4d4-651498fdd061 None admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:57.445133 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a83c6f-ca08-4859-b4d4-651498fdd061 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:57.445425 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a83c6f-ca08-4859-b4d4-651498fdd061 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:57.446131 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a83c6f-ca08-4859-b4d4-651498fdd061 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:57.447520 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a83c6f-ca08-4859-b4d4-651498fdd061 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:57.569284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 530/1072] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:36:57 2022] POST /identity/v3/endpoints => generated 329 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:36:58.184177 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31c62c-dea0-43a4-a484-6ee2d4c0456d None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:58.184837 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31c62c-dea0-43a4-a484-6ee2d4c0456d None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:58.184837 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31c62c-dea0-43a4-a484-6ee2d4c0456d None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:58.186709 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 543/1073] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:58 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:36:58.192417 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05588ff-2b69-4d44-941d-1a1e29cf63df None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:58.192965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05588ff-2b69-4d44-941d-1a1e29cf63df None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:58.192965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05588ff-2b69-4d44-941d-1a1e29cf63df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:58.213397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-f05588ff-2b69-4d44-941d-1a1e29cf63df None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:58.215206 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f05588ff-2b69-4d44-941d-1a1e29cf63df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:58.292000 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 531/1074] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:58 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:36:58.314380 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e31c62c-dea0-43a4-a484-6ee2d4c0456d None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:58.421816 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e31c62c-dea0-43a4-a484-6ee2d4c0456d None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:58.463624 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214d61b-d155-4f05-abc1-05b0a19a6e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:58.464269 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214d61b-d155-4f05-abc1-05b0a19a6e1d None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:58.464552 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214d61b-d155-4f05-abc1-05b0a19a6e1d None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:58.464831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c214d61b-d155-4f05-abc1-05b0a19a6e1d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:58.476137 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c214d61b-d155-4f05-abc1-05b0a19a6e1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:36:58.481904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-c214d61b-d155-4f05-abc1-05b0a19a6e1d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:36:58.481904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:58.481904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:58.481904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:58.481904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:58.481904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:58.481904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:58.481904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:58.481904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:58.481904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:58.481904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:58.481904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:58.481904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:36:58.481904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:36:58.481904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:36:58.481904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:36:58.495401 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:36:58.495401 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:36:58.495401 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:36:58.495401 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:36:58.495401 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:58.495401 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:58.495401 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:58.495401 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:58.495401 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:58.495401 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:58.495401 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:58.495401 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:58.495401 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:58.495401 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:58.495401 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:58.495401 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:58.496971 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:58.496971 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:58.496971 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:58.496971 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:58.496971 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:36:58.496971 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:36:58.496971 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:36:58.496971 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:36:58.496971 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:36:58.496971 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:36:58.496971 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:36:58.496971 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:36:58.496971 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 544/1075] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:36:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:58.497709 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05588ff-2b69-4d44-941d-1a1e29cf63df None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:58.522821 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:58.657597 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05588ff-2b69-4d44-941d-1a1e29cf63df None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:58.695550 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ac203-7bc2-4d4c-b92d-37c1b884cd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:58.696183 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ac203-7bc2-4d4c-b92d-37c1b884cd01 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:58.696460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ac203-7bc2-4d4c-b92d-37c1b884cd01 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:58.696732 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ac203-7bc2-4d4c-b92d-37c1b884cd01 None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:58.697698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ac203-7bc2-4d4c-b92d-37c1b884cd01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:58.698554 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ac203-7bc2-4d4c-b92d-37c1b884cd01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:58.699247 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41ac203-7bc2-4d4c-b92d-37c1b884cd01 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:58.709943 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 532/1076] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:36:58 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:58.716432 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214d61b-d155-4f05-abc1-05b0a19a6e1d None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:58.770870 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:58.829145 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c214d61b-d155-4f05-abc1-05b0a19a6e1d None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:58.863538 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ef660e-ff33-463b-83eb-8efadf204441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:58.864178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef660e-ff33-463b-83eb-8efadf204441 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:58.864460 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef660e-ff33-463b-83eb-8efadf204441 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:58.864735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ef660e-ff33-463b-83eb-8efadf204441 None admin] PATH_INFO: `/v3/users/78e7cd7f62674c558e5c42128ee04634` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:58.865962 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ef660e-ff33-463b-83eb-8efadf204441 None admin] RBAC: Authorizing `identity:get_user(user_id=78e7cd7f62674c558e5c42128ee04634)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:58.867442 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ef660e-ff33-463b-83eb-8efadf204441 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:58.874486 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 545/1077] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:36:58 2022] GET /identity/v3/users/78e7cd7f62674c558e5c42128ee04634 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:58.875871 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ac203-7bc2-4d4c-b92d-37c1b884cd01 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:58.970589 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:58.979625 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41ac203-7bc2-4d4c-b92d-37c1b884cd01 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:59.015851 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4082789a-5f61-4708-86b8-f724b68d550e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:59.016589 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082789a-5f61-4708-86b8-f724b68d550e None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:59.016871 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082789a-5f61-4708-86b8-f724b68d550e None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:59.017149 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4082789a-5f61-4708-86b8-f724b68d550e None admin] PATH_INFO: `/v3/projects/fcb9e5d02067447099445d3a4c0e470b` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:59.018326 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4082789a-5f61-4708-86b8-f724b68d550e None admin] RBAC: Authorizing `identity:get_project(project_id=fcb9e5d02067447099445d3a4c0e470b)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:59.019788 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4082789a-5f61-4708-86b8-f724b68d550e None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:59.027198 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 533/1078] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:36:58 2022] GET /identity/v3/projects/fcb9e5d02067447099445d3a4c0e470b => generated 312 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:59.028234 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ef660e-ff33-463b-83eb-8efadf204441 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:59.104631 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc685883-9c5c-42b2-a3c2-89870e3f9891 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:59.105466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc685883-9c5c-42b2-a3c2-89870e3f9891 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:59.105885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc685883-9c5c-42b2-a3c2-89870e3f9891 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:59.107564 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 534/1079] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:59 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:36:59.112481 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ad06b-d888-4cce-b757-f45463b5ba71 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:59.112767 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ad06b-d888-4cce-b757-f45463b5ba71 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:59.113046 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2ad06b-d888-4cce-b757-f45463b5ba71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:59.132699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-5a2ad06b-d888-4cce-b757-f45463b5ba71 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:59.133903 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-5a2ad06b-d888-4cce-b757-f45463b5ba71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:59.136255 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ef660e-ff33-463b-83eb-8efadf204441 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:59.150012 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:59.172022 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e2134b-9678-4ff3-a4bc-239e29571536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:59.172613 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e2134b-9678-4ff3-a4bc-239e29571536 None admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:59.172915 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e2134b-9678-4ff3-a4bc-239e29571536 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:59.173190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e2134b-9678-4ff3-a4bc-239e29571536 None admin] PATH_INFO: `/v3/projects/fcb9e5d02067447099445d3a4c0e470b/users/78e7cd7f62674c558e5c42128ee04634/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:59.175170 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e2134b-9678-4ff3-a4bc-239e29571536 None admin] RBAC: Authorizing `identity:create_grant(project_id=fcb9e5d02067447099445d3a4c0e470b, user_id=78e7cd7f62674c558e5c42128ee04634, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:59.176596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e2134b-9678-4ff3-a4bc-239e29571536 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:59.216940 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 535/1080] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:59 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:36:59.240492 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ad06b-d888-4cce-b757-f45463b5ba71 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:59.249709 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 546/1081] 10.222.0.115 () {58 vars in 1444 bytes} [Tue Aug 2 10:36:59 2022] PUT /identity/v3/projects/fcb9e5d02067447099445d3a4c0e470b/users/78e7cd7f62674c558e5c42128ee04634/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:36:59.347951 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:59.359500 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2ad06b-d888-4cce-b757-f45463b5ba71 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:59.399034 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b6b50-fda1-448b-a20d-b1ad769a4dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:59.399770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b6b50-fda1-448b-a20d-b1ad769a4dd2 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:59.400050 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b6b50-fda1-448b-a20d-b1ad769a4dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:59.400325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b6b50-fda1-448b-a20d-b1ad769a4dd2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:59.413896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61b6b50-fda1-448b-a20d-b1ad769a4dd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:36:59.419427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-e61b6b50-fda1-448b-a20d-b1ad769a4dd2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:36:59.419427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:59.419427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:59.419427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:59.419427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:59.419427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:59.419427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:59.419427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:59.419427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:59.419427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:59.419427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:59.419427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:59.419427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:36:59.419427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:36:59.419427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:36:59.419427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:36:59.420625 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:36:59.420625 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:36:59.420625 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:36:59.420625 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:36:59.420625 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:59.420625 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:59.420625 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:59.420625 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:59.420625 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:59.420625 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:59.420625 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:59.420625 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:59.420625 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:59.420625 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:59.420625 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:59.420625 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:59.421522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:59.421522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:59.421522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:59.421522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:59.421522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:36:59.421522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:36:59.421522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:36:59.421522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:36:59.421522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:36:59.421522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:36:59.421522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 02 10:36:59.421522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:59.421522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 536/1082] 10.222.0.115 () {56 vars in 1125 bytes} [Tue Aug 2 10:36:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:36:59.426619 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e2134b-9678-4ff3-a4bc-239e29571536 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:59.543757 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e2134b-9678-4ff3-a4bc-239e29571536 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:59.580361 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f74999-e578-4452-a085-c65d0ce4f278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:59.581040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f74999-e578-4452-a085-c65d0ce4f278 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:59.581327 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f74999-e578-4452-a085-c65d0ce4f278 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:59.581602 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f74999-e578-4452-a085-c65d0ce4f278 None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:59.582316 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f74999-e578-4452-a085-c65d0ce4f278 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:59.582622 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f74999-e578-4452-a085-c65d0ce4f278 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:59.584107 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f74999-e578-4452-a085-c65d0ce4f278 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:59.594045 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 547/1083] 10.222.0.115 () {56 vars in 1120 bytes} [Tue Aug 2 10:36:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 340 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:36:59.599463 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b6b50-fda1-448b-a20d-b1ad769a4dd2 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:59.627257 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f907ad-876d-4c7e-bbe4-a9f7fab7cb86 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:59.627571 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f907ad-876d-4c7e-bbe4-a9f7fab7cb86 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:59.627976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f907ad-876d-4c7e-bbe4-a9f7fab7cb86 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:59.629656 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 548/1084] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:36:59 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:36:59.634999 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ab235-36c1-4d98-a2de-ab52a9ddcaf3 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:59.635311 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ab235-36c1-4d98-a2de-ab52a9ddcaf3 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:59.635585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605ab235-36c1-4d98-a2de-ab52a9ddcaf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:59.654643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-605ab235-36c1-4d98-a2de-ab52a9ddcaf3 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:36:59.655950 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-605ab235-36c1-4d98-a2de-ab52a9ddcaf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:36:59.690523 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:59.711492 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b6b50-fda1-448b-a20d-b1ad769a4dd2 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:59.733772 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 549/1085] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:36:59 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:36:59.755631 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ab235-36c1-4d98-a2de-ab52a9ddcaf3 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:59.757440 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acdcb93-24d2-4b6b-a7b6-6b94836be0a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:59.758081 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acdcb93-24d2-4b6b-a7b6-6b94836be0a1 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:59.758385 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acdcb93-24d2-4b6b-a7b6-6b94836be0a1 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:59.758693 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acdcb93-24d2-4b6b-a7b6-6b94836be0a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:59.771694 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acdcb93-24d2-4b6b-a7b6-6b94836be0a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:59.773541 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acdcb93-24d2-4b6b-a7b6-6b94836be0a1 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:59.789892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-9acdcb93-24d2-4b6b-a7b6-6b94836be0a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:59.789892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:36:59.789892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:36:59.789892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:36:59.789892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:36:59.789892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:36:59.789892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:36:59.789892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:36:59.789892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:36:59.789892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:36:59.789892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:36:59.789892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:36:59.789892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:36:59.789892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:36:59.789892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:36:59.790849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:36:59.790849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:36:59.790849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:36:59.790849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:36:59.790849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:36:59.790849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:36:59.790849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:36:59.790849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:36:59.790849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:36:59.790849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:36:59.790849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:36:59.790849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:36:59.790849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:36:59.790849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:36:59.790849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:36:59.790849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:36:59.790849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:36:59.791821 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:36:59.791821 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:36:59.791821 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:36:59.791821 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:36:59.791821 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 537/1086] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:36:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:36:59.796807 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acdcb93-24d2-4b6b-a7b6-6b94836be0a1 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:59.799752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:59.872172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605ab235-36c1-4d98-a2de-ab52a9ddcaf3 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:59.906470 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acdcb93-24d2-4b6b-a7b6-6b94836be0a1 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:36:59.911247 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28234176-90ab-488d-a552-7ca038343fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:59.911878 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28234176-90ab-488d-a552-7ca038343fd0 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:59.912158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28234176-90ab-488d-a552-7ca038343fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:59.912431 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28234176-90ab-488d-a552-7ca038343fd0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:59.942837 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f84ff40-5455-4639-ba4b-decf20833728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:36:59.943524 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84ff40-5455-4639-ba4b-decf20833728 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:36:59.943813 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84ff40-5455-4639-ba4b-decf20833728 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:36:59.944084 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f84ff40-5455-4639-ba4b-decf20833728 None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:36:59.944776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f84ff40-5455-4639-ba4b-decf20833728 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:36:59.945185 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f84ff40-5455-4639-ba4b-decf20833728 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:59.946434 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f84ff40-5455-4639-ba4b-decf20833728 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:36:59.947664 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:36:59.961051 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 538/1087] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:36:59 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:36:59.967824 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f84ff40-5455-4639-ba4b-decf20833728 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:36:59.985702 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28234176-90ab-488d-a552-7ca038343fd0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:36:59.987303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28234176-90ab-488d-a552-7ca038343fd0 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:00.036293 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:00.077970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f84ff40-5455-4639-ba4b-decf20833728 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:00.102056 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 550/1088] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:36:59 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 10:37:00.108478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28234176-90ab-488d-a552-7ca038343fd0 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:00.122584 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055b121-94dd-4aad-8ddb-af44df0cca7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:00.123264 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055b121-94dd-4aad-8ddb-af44df0cca7e None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:00.123599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055b121-94dd-4aad-8ddb-af44df0cca7e None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:00.123804 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d055b121-94dd-4aad-8ddb-af44df0cca7e None admin] PATH_INFO: `/v3/users/nova` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:00.132070 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d055b121-94dd-4aad-8ddb-af44df0cca7e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:00.133510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d055b121-94dd-4aad-8ddb-af44df0cca7e None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:00.146930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-d055b121-94dd-4aad-8ddb-af44df0cca7e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:37:00.146930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:37:00.146930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:37:00.146930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:37:00.146930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:37:00.146930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:37:00.146930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:37:00.146930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:37:00.146930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:37:00.146930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:37:00.146930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:37:00.146930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:37:00.146930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:37:00.146930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:37:00.146930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:37:00.146930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:37:00.148127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:37:00.148127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:37:00.148127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:37:00.148127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:37:00.148127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:37:00.148127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:37:00.148127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:37:00.148127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:37:00.148127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:37:00.148127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:37:00.148127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:37:00.148127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:37:00.148127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:37:00.148127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:37:00.148127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:37:00.148127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:37:00.148127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:37:00.148127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:37:00.149442 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:37:00.149442 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:37:00.149442 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:37:00.149442 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:37:00.149442 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:37:00.149442 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:37:00.149442 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:37:00.149442 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:37:00.149442 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 02 10:37:00.149442 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:37:00.149442 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 539/1089] 10.222.0.115 () {56 vars in 1098 bytes} [Tue Aug 2 10:36:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:37:00.154579 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055b121-94dd-4aad-8ddb-af44df0cca7e None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:00.183952 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:00.237349 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28234176-90ab-488d-a552-7ca038343fd0 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:00.273982 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d055b121-94dd-4aad-8ddb-af44df0cca7e None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:00.274550 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf26623-4d5e-4923-a4d2-29a0de680413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:00.275071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf26623-4d5e-4923-a4d2-29a0de680413 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:00.275357 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf26623-4d5e-4923-a4d2-29a0de680413 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:00.275636 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf26623-4d5e-4923-a4d2-29a0de680413 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:00.288765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf26623-4d5e-4923-a4d2-29a0de680413 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:00.290228 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf26623-4d5e-4923-a4d2-29a0de680413 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:00.301578 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:00.306393 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-3bf26623-4d5e-4923-a4d2-29a0de680413 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:37:00.306393 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:37:00.306393 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:37:00.306393 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:37:00.306393 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:37:00.306393 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:37:00.306393 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:37:00.306393 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:37:00.306393 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:37:00.306393 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:37:00.306393 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:37:00.306393 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:37:00.306393 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:37:00.306393 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:37:00.306393 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:37:00.307437 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:37:00.307437 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:37:00.307437 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:37:00.307437 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:37:00.307437 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:37:00.307437 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:37:00.307437 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:37:00.307437 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:37:00.307437 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:37:00.307437 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:37:00.307437 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:37:00.307437 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:37:00.307437 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:37:00.307437 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:37:00.307437 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:37:00.307437 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:37:00.307437 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:37:00.309022 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:37:00.309022 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:37:00.309022 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:37:00.309022 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:37:00.313186 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 551/1090] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:37:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:37:00.314349 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf26623-4d5e-4923-a4d2-29a0de680413 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:00.316139 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ffdee-35af-49f5-9b99-a7c36c1baffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:00.323092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ffdee-35af-49f5-9b99-a7c36c1baffc None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:00.323092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ffdee-35af-49f5-9b99-a7c36c1baffc None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:00.323092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5ffdee-35af-49f5-9b99-a7c36c1baffc None admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:00.323092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5ffdee-35af-49f5-9b99-a7c36c1baffc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:37:00.323092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5ffdee-35af-49f5-9b99-a7c36c1baffc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:00.323092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5ffdee-35af-49f5-9b99-a7c36c1baffc None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:00.343180 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 540/1091] 10.222.0.115 () {56 vars in 1138 bytes} [Tue Aug 2 10:37:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:37:00.348622 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ffdee-35af-49f5-9b99-a7c36c1baffc None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:00.438786 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf26623-4d5e-4923-a4d2-29a0de680413 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:00.459351 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:00.472441 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5ffdee-35af-49f5-9b99-a7c36c1baffc None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:00.478761 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24956b-0169-4ea1-88e3-708e50fa60c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:00.488934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24956b-0169-4ea1-88e3-708e50fa60c2 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:00.488934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24956b-0169-4ea1-88e3-708e50fa60c2 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:00.488934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b24956b-0169-4ea1-88e3-708e50fa60c2 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:00.488934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24956b-0169-4ea1-88e3-708e50fa60c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:37:00.488934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24956b-0169-4ea1-88e3-708e50fa60c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:00.488934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b24956b-0169-4ea1-88e3-708e50fa60c2 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:00.497349 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 552/1092] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:37:00 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:37:00.515594 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c1fa5-d718-45e4-8fae-3ee0c1b5e3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:00.516313 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c1fa5-d718-45e4-8fae-3ee0c1b5e3b9 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:00.522140 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c1fa5-d718-45e4-8fae-3ee0c1b5e3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:00.522140 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070c1fa5-d718-45e4-8fae-3ee0c1b5e3b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:00.578498 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:00.601997 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c1fa5-d718-45e4-8fae-3ee0c1b5e3b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:00.603468 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070c1fa5-d718-45e4-8fae-3ee0c1b5e3b9 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:00.731331 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 541/1093] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:37:00 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 10:37:00.739004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24956b-0169-4ea1-88e3-708e50fa60c2 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:00.856039 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b24956b-0169-4ea1-88e3-708e50fa60c2 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:00.892926 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8cf61-bcbd-400f-bc85-0795302aafcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:00.893574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8cf61-bcbd-400f-bc85-0795302aafcc None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:00.893858 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8cf61-bcbd-400f-bc85-0795302aafcc None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:00.894127 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab8cf61-bcbd-400f-bc85-0795302aafcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:00.904683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8cf61-bcbd-400f-bc85-0795302aafcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:00.906122 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab8cf61-bcbd-400f-bc85-0795302aafcc None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:00.922178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-0ab8cf61-bcbd-400f-bc85-0795302aafcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:37:00.922178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:37:00.922178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:37:00.922178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:37:00.922178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:37:00.922178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:37:00.922178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:37:00.922178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:37:00.922178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:37:00.922178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:37:00.922178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:37:00.922178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:37:00.922178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:37:00.922178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:37:00.922178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:37:00.923207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:37:00.923207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:37:00.923207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:37:00.923207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:37:00.923207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:37:00.923207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:37:00.923207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:37:00.923207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:37:00.923207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:37:00.923207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:37:00.923207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:37:00.923207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:37:00.923207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:37:00.923207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:37:00.923207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:37:00.923207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:37:00.923207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:37:00.924138 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:37:00.924138 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:37:00.924138 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:37:00.924138 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:37:00.924138 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 553/1094] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:37:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:37:00.930762 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c1fa5-d718-45e4-8fae-3ee0c1b5e3b9 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:01.022488 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:01.043827 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070c1fa5-d718-45e4-8fae-3ee0c1b5e3b9 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:01.083682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb08a3c-7ac6-4029-bda9-45e5981b0003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:01.084297 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb08a3c-7ac6-4029-bda9-45e5981b0003 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:01.084571 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb08a3c-7ac6-4029-bda9-45e5981b0003 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:01.084841 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb08a3c-7ac6-4029-bda9-45e5981b0003 None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:01.085531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb08a3c-7ac6-4029-bda9-45e5981b0003 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:37:01.085831 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb08a3c-7ac6-4029-bda9-45e5981b0003 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:01.087230 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb08a3c-7ac6-4029-bda9-45e5981b0003 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:01.099194 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 542/1095] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:37:00 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:01.104996 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8cf61-bcbd-400f-bc85-0795302aafcc None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:01.138366 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:01.210516 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab8cf61-bcbd-400f-bc85-0795302aafcc None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:01.248675 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306b4e5e-fa86-4021-b638-97e1860dbcb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:01.249325 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b4e5e-fa86-4021-b638-97e1860dbcb0 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:01.249607 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b4e5e-fa86-4021-b638-97e1860dbcb0 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:01.249881 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306b4e5e-fa86-4021-b638-97e1860dbcb0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:01.256953 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b4e5e-fa86-4021-b638-97e1860dbcb0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:01.258384 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-306b4e5e-fa86-4021-b638-97e1860dbcb0 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:01.270488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-306b4e5e-fa86-4021-b638-97e1860dbcb0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:37:01.270488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:37:01.270488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:37:01.270488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:37:01.270488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:37:01.270488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:37:01.270488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:37:01.270488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:37:01.270488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:37:01.270488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:37:01.270488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:37:01.270488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:37:01.270488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:37:01.270488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:37:01.270488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:37:01.271478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:37:01.271478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:37:01.271478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:37:01.271478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:37:01.271478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:37:01.271478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:37:01.271478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:37:01.271478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:37:01.271478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:37:01.271478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:37:01.271478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:37:01.271478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:37:01.271478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:37:01.271478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:37:01.271478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:37:01.271478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:37:01.271478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:37:01.272405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:37:01.272405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:37:01.272405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:37:01.272405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:37:01.272405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:37:01.272405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:37:01.272405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:37:01.272405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:37:01.272405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 554/1096] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:37:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:37:01.278988 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb08a3c-7ac6-4029-bda9-45e5981b0003 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:01.349709 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f5145-5a94-45ac-82b2-152562ad4693 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:01.350241 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f5145-5a94-45ac-82b2-152562ad4693 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:01.350241 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0f5145-5a94-45ac-82b2-152562ad4693 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:01.351655 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 555/1097] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:37:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:37:01.356932 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84914202-6388-445a-9ade-650b9cdf8242 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:01.357217 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84914202-6388-445a-9ade-650b9cdf8242 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:01.357494 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84914202-6388-445a-9ade-650b9cdf8242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:01.375778 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-84914202-6388-445a-9ade-650b9cdf8242 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:37:01.376966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-84914202-6388-445a-9ade-650b9cdf8242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:37:01.382892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb08a3c-7ac6-4029-bda9-45e5981b0003 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:01.418396 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a39af-8037-469d-9280-fa8fd5ff3ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:01.418894 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a39af-8037-469d-9280-fa8fd5ff3ce3 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:01.419208 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a39af-8037-469d-9280-fa8fd5ff3ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:01.419489 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a39af-8037-469d-9280-fa8fd5ff3ce3 None admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:01.420242 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a39af-8037-469d-9280-fa8fd5ff3ce3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:37:01.420550 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a39af-8037-469d-9280-fa8fd5ff3ce3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:01.421987 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4a39af-8037-469d-9280-fa8fd5ff3ce3 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:01.431537 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:01.437541 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 543/1098] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:37:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:37:01.443737 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a39af-8037-469d-9280-fa8fd5ff3ce3 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:01.453488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 556/1099] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:37:01 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:37:01.476503 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84914202-6388-445a-9ade-650b9cdf8242 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:01.499626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:01.547938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4a39af-8037-469d-9280-fa8fd5ff3ce3 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:01.579071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84914202-6388-445a-9ade-650b9cdf8242 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:01.583911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b965e5-ea4d-4ae0-9587-95b9915c9dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:01.584541 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b965e5-ea4d-4ae0-9587-95b9915c9dae None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:01.584841 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b965e5-ea4d-4ae0-9587-95b9915c9dae None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:01.585140 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b965e5-ea4d-4ae0-9587-95b9915c9dae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:01.585982 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b965e5-ea4d-4ae0-9587-95b9915c9dae None admin] RBAC: Adding query filter params (role.id=7fee85e1d15545a1926ac22137e06585, scope.project.id=c1eae881e5714bf4bd223809e1a2d920, user.id=d0478bd540f446ec810e2091c46ac49c) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:37:01.586291 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b965e5-ea4d-4ae0-9587-95b9915c9dae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:01.587814 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b965e5-ea4d-4ae0-9587-95b9915c9dae None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:01.595846 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 544/1100] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:37:01 2022] GET /identity/v3/role_assignments?user.id=d0478bd540f446ec810e2091c46ac49c&scope.project.id=c1eae881e5714bf4bd223809e1a2d920&role.id=7fee85e1d15545a1926ac22137e06585 => generated 610 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:37:01.632475 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c33a33-c162-45ae-a282-c09c16404c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:01.633140 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c33a33-c162-45ae-a282-c09c16404c1c None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:01.633422 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c33a33-c162-45ae-a282-c09c16404c1c None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:01.633694 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c33a33-c162-45ae-a282-c09c16404c1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:01.647757 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c33a33-c162-45ae-a282-c09c16404c1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:37:01.652928 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-b6c33a33-c162-45ae-a282-c09c16404c1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:37:01.652928 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:37:01.652928 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:37:01.652928 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:37:01.652928 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:37:01.652928 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:37:01.652928 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:37:01.652928 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:37:01.652928 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:37:01.652928 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:37:01.652928 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:37:01.652928 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:37:01.652928 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:37:01.652928 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:37:01.652928 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:37:01.652928 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:37:01.654174 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:37:01.654174 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:37:01.654174 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:37:01.654174 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:37:01.654174 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:37:01.654174 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:37:01.654174 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:37:01.654174 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:37:01.654174 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:37:01.654174 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:37:01.654174 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:37:01.654174 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:37:01.654174 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:37:01.654174 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:37:01.654174 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:37:01.654174 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:37:01.655419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:37:01.655419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:37:01.655419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:37:01.655419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:37:01.655419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:37:01.655419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:37:01.655419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:37:01.655419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:37:01.655419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:37:01.655419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:37:01.655419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:37:01.655419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:37:01.659298 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 557/1101] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:37:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:37:01.660673 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b965e5-ea4d-4ae0-9587-95b9915c9dae None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:01.731314 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:01.774205 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b965e5-ea4d-4ae0-9587-95b9915c9dae None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:01.814522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173734d0-7583-497d-92ac-7bf3b0677d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:01.815191 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173734d0-7583-497d-92ac-7bf3b0677d7b None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:01.815474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173734d0-7583-497d-92ac-7bf3b0677d7b None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:01.815750 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173734d0-7583-497d-92ac-7bf3b0677d7b None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:01.816520 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173734d0-7583-497d-92ac-7bf3b0677d7b None admin] RBAC: Adding query filter params (name=admin) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:37:01.816997 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173734d0-7583-497d-92ac-7bf3b0677d7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:01.818369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173734d0-7583-497d-92ac-7bf3b0677d7b None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:01.829448 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 545/1102] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:37:01 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:37:01.836011 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c33a33-c162-45ae-a282-c09c16404c1c None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:01.930538 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:01.952268 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c33a33-c162-45ae-a282-c09c16404c1c None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:01.991317 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df002b8-64a8-4e16-a4cb-a0a1c372dd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:01.991970 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df002b8-64a8-4e16-a4cb-a0a1c372dd3c None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:01.992343 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df002b8-64a8-4e16-a4cb-a0a1c372dd3c None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:01.992689 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df002b8-64a8-4e16-a4cb-a0a1c372dd3c None admin] PATH_INFO: `/v3/users/78e7cd7f62674c558e5c42128ee04634` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:01.994254 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df002b8-64a8-4e16-a4cb-a0a1c372dd3c None admin] RBAC: Authorizing `identity:get_user(user_id=78e7cd7f62674c558e5c42128ee04634)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:01.997160 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df002b8-64a8-4e16-a4cb-a0a1c372dd3c None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:01.999397 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 558/1103] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:37:01 2022] GET /identity/v3/users/78e7cd7f62674c558e5c42128ee04634 => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:02.007024 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173734d0-7583-497d-92ac-7bf3b0677d7b None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:02.116608 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173734d0-7583-497d-92ac-7bf3b0677d7b None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:02.145156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:02.154845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e4f6e-33ac-4594-9e9c-de25e5d6f114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:02.155523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e4f6e-33ac-4594-9e9c-de25e5d6f114 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:02.155799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e4f6e-33ac-4594-9e9c-de25e5d6f114 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:02.158600 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3e4f6e-33ac-4594-9e9c-de25e5d6f114 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:02.230913 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3e4f6e-33ac-4594-9e9c-de25e5d6f114 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:02.232476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3e4f6e-33ac-4594-9e9c-de25e5d6f114 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:02.342665 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 546/1104] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:37:02 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:37:02.348584 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df002b8-64a8-4e16-a4cb-a0a1c372dd3c None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:02.458893 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df002b8-64a8-4e16-a4cb-a0a1c372dd3c None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:02.496386 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e58947-1d76-4164-860a-a5d188a250b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:02.496386 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e58947-1d76-4164-860a-a5d188a250b4 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:02.497025 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e58947-1d76-4164-860a-a5d188a250b4 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:02.503495 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e58947-1d76-4164-860a-a5d188a250b4 None admin] PATH_INFO: `/v3/projects/fcb9e5d02067447099445d3a4c0e470b` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:02.503495 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e58947-1d76-4164-860a-a5d188a250b4 None admin] RBAC: Authorizing `identity:get_project(project_id=fcb9e5d02067447099445d3a4c0e470b)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:02.503495 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e58947-1d76-4164-860a-a5d188a250b4 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:02.503495 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 559/1105] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:37:02 2022] GET /identity/v3/projects/fcb9e5d02067447099445d3a4c0e470b => generated 312 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:02.507852 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e4f6e-33ac-4594-9e9c-de25e5d6f114 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:02.536630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:02.619016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3e4f6e-33ac-4594-9e9c-de25e5d6f114 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:02.652277 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69336d63-a951-4346-9aaa-82e4e7153cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:02.653022 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69336d63-a951-4346-9aaa-82e4e7153cb5 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:02.653298 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69336d63-a951-4346-9aaa-82e4e7153cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:02.653564 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69336d63-a951-4346-9aaa-82e4e7153cb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:02.654348 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69336d63-a951-4346-9aaa-82e4e7153cb5 None admin] RBAC: Adding query filter params (role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac, scope.project.id=fcb9e5d02067447099445d3a4c0e470b, user.id=78e7cd7f62674c558e5c42128ee04634) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:37:02.654651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69336d63-a951-4346-9aaa-82e4e7153cb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:02.656101 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69336d63-a951-4346-9aaa-82e4e7153cb5 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:02.661939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 547/1106] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:37:02 2022] GET /identity/v3/role_assignments?user.id=78e7cd7f62674c558e5c42128ee04634&scope.project.id=fcb9e5d02067447099445d3a4c0e470b&role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 610 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:04.698188 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e20e7-eb79-4dde-bc53-b6a2bde51997 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:04.698869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e20e7-eb79-4dde-bc53-b6a2bde51997 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:04.698966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4e20e7-eb79-4dde-bc53-b6a2bde51997 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:04.700336 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 560/1107] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:37:04 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:37:04.706591 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333b6af-ffa8-4e94-b1b8-51fd08b3d95e None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:04.707115 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333b6af-ffa8-4e94-b1b8-51fd08b3d95e None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:04.707230 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9333b6af-ffa8-4e94-b1b8-51fd08b3d95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:04.725493 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-9333b6af-ffa8-4e94-b1b8-51fd08b3d95e None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:37:04.726682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-9333b6af-ffa8-4e94-b1b8-51fd08b3d95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:37:04.795931 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 548/1108] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:37:04 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:37:04.820161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e20e7-eb79-4dde-bc53-b6a2bde51997 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:04.841334 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:04.927373 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4e20e7-eb79-4dde-bc53-b6a2bde51997 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:04.962367 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e380f-ea0a-4656-9ef5-99c83da52600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:04.962902 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e380f-ea0a-4656-9ef5-99c83da52600 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:04.963202 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e380f-ea0a-4656-9ef5-99c83da52600 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:04.963470 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e380f-ea0a-4656-9ef5-99c83da52600 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:04.965466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e380f-ea0a-4656-9ef5-99c83da52600 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:04.966889 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e380f-ea0a-4656-9ef5-99c83da52600 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:04.968592 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 561/1109] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:37:04 2022] GET /identity/v3/domains/default => generated 200 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:37:04.974397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9333b6af-ffa8-4e94-b1b8-51fd08b3d95e None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:04.995900 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:05.074796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9333b6af-ffa8-4e94-b1b8-51fd08b3d95e None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:05.109156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294431f8-3bf2-44d8-8738-dbc8a966497a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:05.109759 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294431f8-3bf2-44d8-8738-dbc8a966497a None admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:05.110037 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294431f8-3bf2-44d8-8738-dbc8a966497a None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:05.110310 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294431f8-3bf2-44d8-8738-dbc8a966497a None admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:05.110998 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294431f8-3bf2-44d8-8738-dbc8a966497a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:05.112445 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294431f8-3bf2-44d8-8738-dbc8a966497a None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:05.225833 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 549/1110] 10.222.0.115 () {60 vars in 1137 bytes} [Tue Aug 2 10:37:04 2022] POST /identity/v3/users => generated 287 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:37:07.279950 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd070fba-aa5e-488b-9ff1-d9ad88f3062c None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:07.281330 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd070fba-aa5e-488b-9ff1-d9ad88f3062c None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:07.281330 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd070fba-aa5e-488b-9ff1-d9ad88f3062c None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:07.281872 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 562/1111] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:37:07 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:37:07.287195 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922068b0-fffe-403b-ab30-ac615a52d675 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:07.287623 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922068b0-fffe-403b-ab30-ac615a52d675 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:07.287740 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922068b0-fffe-403b-ab30-ac615a52d675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:07.306400 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-922068b0-fffe-403b-ab30-ac615a52d675 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:37:07.307755 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-922068b0-fffe-403b-ab30-ac615a52d675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:37:07.375923 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 550/1112] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:37:07 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:37:07.399239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd070fba-aa5e-488b-9ff1-d9ad88f3062c None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:07.487804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:07.504899 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd070fba-aa5e-488b-9ff1-d9ad88f3062c None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:07.539603 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e601b2-fd00-4593-a77c-2d5004ef330b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:07.540158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e601b2-fd00-4593-a77c-2d5004ef330b None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:07.540435 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e601b2-fd00-4593-a77c-2d5004ef330b None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:07.540706 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e601b2-fd00-4593-a77c-2d5004ef330b None admin] PATH_INFO: `/v3/roles/a4213d713b6f49d78d7bbb9484c9312f` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:07.542643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e601b2-fd00-4593-a77c-2d5004ef330b None admin] RBAC: Authorizing `identity:get_role(role_id=a4213d713b6f49d78d7bbb9484c9312f)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:07.544380 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e601b2-fd00-4593-a77c-2d5004ef330b None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:07.545731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 563/1113] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:37:07 2022] GET /identity/v3/roles/a4213d713b6f49d78d7bbb9484c9312f => generated 224 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:07.551831 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922068b0-fffe-403b-ab30-ac615a52d675 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:07.561139 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:07.654250 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922068b0-fffe-403b-ab30-ac615a52d675 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:07.688589 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f0c8c-0c4a-4556-8cc4-1fb1e85dcf51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:07.689222 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f0c8c-0c4a-4556-8cc4-1fb1e85dcf51 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:07.689495 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f0c8c-0c4a-4556-8cc4-1fb1e85dcf51 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:07.689764 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f0c8c-0c4a-4556-8cc4-1fb1e85dcf51 None admin] PATH_INFO: `/v3/users/b41a9018f7564c729ffdb8838010138b` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:07.703750 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9f0c8c-0c4a-4556-8cc4-1fb1e85dcf51 None admin] RBAC: Authorizing `identity:get_user(user_id=b41a9018f7564c729ffdb8838010138b)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:07.705176 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9f0c8c-0c4a-4556-8cc4-1fb1e85dcf51 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:07.706861 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 551/1114] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:37:07 2022] GET /identity/v3/users/b41a9018f7564c729ffdb8838010138b => generated 287 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:07.712857 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e601b2-fd00-4593-a77c-2d5004ef330b None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:07.823263 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e601b2-fd00-4593-a77c-2d5004ef330b None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:07.859014 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fedcc96-247a-40ce-a78f-5f3d4b683fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:07.859651 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fedcc96-247a-40ce-a78f-5f3d4b683fc1 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:07.859927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fedcc96-247a-40ce-a78f-5f3d4b683fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:07.860195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fedcc96-247a-40ce-a78f-5f3d4b683fc1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:07.881361 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:07.929341 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fedcc96-247a-40ce-a78f-5f3d4b683fc1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:07.930810 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fedcc96-247a-40ce-a78f-5f3d4b683fc1 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:08.049635 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 564/1115] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:37:07 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:37:08.058922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f0c8c-0c4a-4556-8cc4-1fb1e85dcf51 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:08.124894 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:08.164141 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f0c8c-0c4a-4556-8cc4-1fb1e85dcf51 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:08.198147 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5228e15d-0e79-4687-9cc9-7e5558675797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:08.199525 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228e15d-0e79-4687-9cc9-7e5558675797 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:08.199869 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228e15d-0e79-4687-9cc9-7e5558675797 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:08.200145 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5228e15d-0e79-4687-9cc9-7e5558675797 None admin] PATH_INFO: `/v3/projects/fcb9e5d02067447099445d3a4c0e470b` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:08.201305 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5228e15d-0e79-4687-9cc9-7e5558675797 None admin] RBAC: Authorizing `identity:get_project(project_id=fcb9e5d02067447099445d3a4c0e470b)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:08.202720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5228e15d-0e79-4687-9cc9-7e5558675797 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:08.204670 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 552/1116] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:37:08 2022] GET /identity/v3/projects/fcb9e5d02067447099445d3a4c0e470b => generated 312 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:08.214714 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fedcc96-247a-40ce-a78f-5f3d4b683fc1 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:08.275696 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:08.322955 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fedcc96-247a-40ce-a78f-5f3d4b683fc1 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:08.359159 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6c517f-cea9-4edb-b233-f44d32f70ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:08.359761 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c517f-cea9-4edb-b233-f44d32f70ae4 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:08.360039 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c517f-cea9-4edb-b233-f44d32f70ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:08.360306 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c517f-cea9-4edb-b233-f44d32f70ae4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:08.361081 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c517f-cea9-4edb-b233-f44d32f70ae4 None admin] RBAC: Adding query filter params (role.id=a4213d713b6f49d78d7bbb9484c9312f, scope.project.id=fcb9e5d02067447099445d3a4c0e470b, user.id=b41a9018f7564c729ffdb8838010138b) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:37:08.361394 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c517f-cea9-4edb-b233-f44d32f70ae4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:08.362801 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6c517f-cea9-4edb-b233-f44d32f70ae4 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:08.368060 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 565/1117] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:37:08 2022] GET /identity/v3/role_assignments?user.id=b41a9018f7564c729ffdb8838010138b&scope.project.id=fcb9e5d02067447099445d3a4c0e470b&role.id=a4213d713b6f49d78d7bbb9484c9312f => generated 260 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:10.361336 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa09b9-d413-4838-8deb-58e05a0f5ad5 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:10.361902 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa09b9-d413-4838-8deb-58e05a0f5ad5 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:10.361902 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebaa09b9-d413-4838-8deb-58e05a0f5ad5 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:10.363435 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 553/1118] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:37:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:37:10.368795 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af637f4-23d7-4933-9d81-976394a3521b None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:10.369239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af637f4-23d7-4933-9d81-976394a3521b None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:10.369359 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af637f4-23d7-4933-9d81-976394a3521b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:10.388443 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-2af637f4-23d7-4933-9d81-976394a3521b None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:37:10.389662 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-2af637f4-23d7-4933-9d81-976394a3521b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:37:10.465699 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 566/1119] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:37:10 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:37:10.488608 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa09b9-d413-4838-8deb-58e05a0f5ad5 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:10.598297 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebaa09b9-d413-4838-8deb-58e05a0f5ad5 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:10.631413 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:10.639165 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd921e-b0cb-46ba-ac0f-781770946a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:10.639803 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd921e-b0cb-46ba-ac0f-781770946a5b None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:10.640085 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd921e-b0cb-46ba-ac0f-781770946a5b None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:10.640362 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd921e-b0cb-46ba-ac0f-781770946a5b None admin] PATH_INFO: `/v3/roles/a4213d713b6f49d78d7bbb9484c9312f` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:10.642230 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd921e-b0cb-46ba-ac0f-781770946a5b None admin] RBAC: Authorizing `identity:get_role(role_id=a4213d713b6f49d78d7bbb9484c9312f)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:10.643662 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbd921e-b0cb-46ba-ac0f-781770946a5b None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:10.645070 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 554/1120] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:37:10 2022] GET /identity/v3/roles/a4213d713b6f49d78d7bbb9484c9312f => generated 224 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:37:10.651024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af637f4-23d7-4933-9d81-976394a3521b None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:10.702580 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:10.760469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af637f4-23d7-4933-9d81-976394a3521b None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:10.796757 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85307b19-251e-4be6-b11e-b5785cc9ded6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:10.797396 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85307b19-251e-4be6-b11e-b5785cc9ded6 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:10.797679 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85307b19-251e-4be6-b11e-b5785cc9ded6 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:10.797948 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85307b19-251e-4be6-b11e-b5785cc9ded6 None admin] PATH_INFO: `/v3/users/b41a9018f7564c729ffdb8838010138b` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:10.799127 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85307b19-251e-4be6-b11e-b5785cc9ded6 None admin] RBAC: Authorizing `identity:get_user(user_id=b41a9018f7564c729ffdb8838010138b)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:10.800549 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85307b19-251e-4be6-b11e-b5785cc9ded6 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:10.802154 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 567/1121] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:37:10 2022] GET /identity/v3/users/b41a9018f7564c729ffdb8838010138b => generated 287 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:10.808696 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd921e-b0cb-46ba-ac0f-781770946a5b None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:10.909433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd921e-b0cb-46ba-ac0f-781770946a5b None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:10.943823 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5150c4b0-9f15-4925-aaf9-454350aef260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:10.944466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150c4b0-9f15-4925-aaf9-454350aef260 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:10.944746 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150c4b0-9f15-4925-aaf9-454350aef260 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:10.945023 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150c4b0-9f15-4925-aaf9-454350aef260 None admin] PATH_INFO: `/v3/projects/fcb9e5d02067447099445d3a4c0e470b` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:10.946183 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5150c4b0-9f15-4925-aaf9-454350aef260 None admin] RBAC: Authorizing `identity:get_project(project_id=fcb9e5d02067447099445d3a4c0e470b)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:10.947681 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5150c4b0-9f15-4925-aaf9-454350aef260 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:10.950107 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 555/1122] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:37:10 2022] GET /identity/v3/projects/fcb9e5d02067447099445d3a4c0e470b => generated 312 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:10.956502 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85307b19-251e-4be6-b11e-b5785cc9ded6 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:11.070019 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85307b19-251e-4be6-b11e-b5785cc9ded6 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:11.085486 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:11.106599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14bce87-c975-4ce6-a254-5146103a2b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:11.107310 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14bce87-c975-4ce6-a254-5146103a2b8a None admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:11.107591 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14bce87-c975-4ce6-a254-5146103a2b8a None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:11.107865 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14bce87-c975-4ce6-a254-5146103a2b8a None admin] PATH_INFO: `/v3/projects/fcb9e5d02067447099445d3a4c0e470b/users/b41a9018f7564c729ffdb8838010138b/roles/a4213d713b6f49d78d7bbb9484c9312f` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:11.110140 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14bce87-c975-4ce6-a254-5146103a2b8a None admin] RBAC: Authorizing `identity:create_grant(project_id=fcb9e5d02067447099445d3a4c0e470b, user_id=b41a9018f7564c729ffdb8838010138b, role_id=a4213d713b6f49d78d7bbb9484c9312f)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:11.111597 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14bce87-c975-4ce6-a254-5146103a2b8a None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:11.173585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 568/1123] 10.222.0.115 () {58 vars in 1444 bytes} [Tue Aug 2 10:37:10 2022] PUT /identity/v3/projects/fcb9e5d02067447099445d3a4c0e470b/users/b41a9018f7564c729ffdb8838010138b/roles/a4213d713b6f49d78d7bbb9484c9312f => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:37:13.223075 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44a979-c82f-43f5-b758-8dee2efa784f None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:13.223647 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44a979-c82f-43f5-b758-8dee2efa784f None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:13.223647 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b44a979-c82f-43f5-b758-8dee2efa784f None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:13.224988 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 556/1124] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:37:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:37:13.231157 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6236a-97a4-40a9-a460-bdf251dd8b25 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:13.231724 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6236a-97a4-40a9-a460-bdf251dd8b25 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:13.231724 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6236a-97a4-40a9-a460-bdf251dd8b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:13.251508 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-1ae6236a-97a4-40a9-a460-bdf251dd8b25 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:37:13.252692 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-1ae6236a-97a4-40a9-a460-bdf251dd8b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:37:13.322737 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 569/1125] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:37:13 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:37:13.348200 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44a979-c82f-43f5-b758-8dee2efa784f None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:13.419830 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:13.460837 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b44a979-c82f-43f5-b758-8dee2efa784f None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:13.496171 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e985b26c-7dd3-44f9-a5e0-4a63a3b773bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:13.496759 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985b26c-7dd3-44f9-a5e0-4a63a3b773bd None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:13.497036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985b26c-7dd3-44f9-a5e0-4a63a3b773bd None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:13.497308 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985b26c-7dd3-44f9-a5e0-4a63a3b773bd None admin] PATH_INFO: `/v3/roles/a4213d713b6f49d78d7bbb9484c9312f` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:13.498764 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e985b26c-7dd3-44f9-a5e0-4a63a3b773bd None admin] RBAC: Authorizing `identity:get_role(role_id=a4213d713b6f49d78d7bbb9484c9312f)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:13.500154 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e985b26c-7dd3-44f9-a5e0-4a63a3b773bd None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:13.501510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 557/1126] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:37:13 2022] GET /identity/v3/roles/a4213d713b6f49d78d7bbb9484c9312f => generated 224 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:13.507466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae6236a-97a4-40a9-a460-bdf251dd8b25 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:13.607972 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae6236a-97a4-40a9-a460-bdf251dd8b25 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:13.610328 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:13.642443 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77f5a2-4d6c-465a-97d7-9a58777abe74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:13.643095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77f5a2-4d6c-465a-97d7-9a58777abe74 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:13.643379 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77f5a2-4d6c-465a-97d7-9a58777abe74 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:13.643658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77f5a2-4d6c-465a-97d7-9a58777abe74 None admin] PATH_INFO: `/v3/users/b41a9018f7564c729ffdb8838010138b` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:13.644809 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd77f5a2-4d6c-465a-97d7-9a58777abe74 None admin] RBAC: Authorizing `identity:get_user(user_id=b41a9018f7564c729ffdb8838010138b)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:13.646247 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd77f5a2-4d6c-465a-97d7-9a58777abe74 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:13.648738 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 570/1127] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:37:13 2022] GET /identity/v3/users/b41a9018f7564c729ffdb8838010138b => generated 287 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:13.653982 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e985b26c-7dd3-44f9-a5e0-4a63a3b773bd None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:13.763156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e985b26c-7dd3-44f9-a5e0-4a63a3b773bd None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:13.796170 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ee55d-c802-41e0-baeb-86e4c9a07a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:13.796763 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ee55d-c802-41e0-baeb-86e4c9a07a51 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:13.797039 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ee55d-c802-41e0-baeb-86e4c9a07a51 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:13.797302 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ee55d-c802-41e0-baeb-86e4c9a07a51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:13.800715 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:13.864428 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577ee55d-c802-41e0-baeb-86e4c9a07a51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:13.865873 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577ee55d-c802-41e0-baeb-86e4c9a07a51 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:13.965401 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 558/1128] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:37:13 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:37:13.972694 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77f5a2-4d6c-465a-97d7-9a58777abe74 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:14.076845 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77f5a2-4d6c-465a-97d7-9a58777abe74 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:14.111500 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609a7a4d-0637-4fab-9cda-da620fbe2d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:14.112151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a7a4d-0637-4fab-9cda-da620fbe2d2c None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:14.112434 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a7a4d-0637-4fab-9cda-da620fbe2d2c None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:14.112710 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609a7a4d-0637-4fab-9cda-da620fbe2d2c None admin] PATH_INFO: `/v3/projects/fcb9e5d02067447099445d3a4c0e470b` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:14.113868 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a7a4d-0637-4fab-9cda-da620fbe2d2c None admin] RBAC: Authorizing `identity:get_project(project_id=fcb9e5d02067447099445d3a4c0e470b)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:14.115325 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609a7a4d-0637-4fab-9cda-da620fbe2d2c None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:14.116985 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 571/1129] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:37:13 2022] GET /identity/v3/projects/fcb9e5d02067447099445d3a4c0e470b => generated 312 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:14.124163 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ee55d-c802-41e0-baeb-86e4c9a07a51 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:14.157709 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:14.223065 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ee55d-c802-41e0-baeb-86e4c9a07a51 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:14.257719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5351234c-5461-4df2-88a3-2d6305eb4a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:14.258369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351234c-5461-4df2-88a3-2d6305eb4a8a None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:14.258583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351234c-5461-4df2-88a3-2d6305eb4a8a None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:14.258854 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5351234c-5461-4df2-88a3-2d6305eb4a8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:14.259661 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351234c-5461-4df2-88a3-2d6305eb4a8a None admin] RBAC: Adding query filter params (role.id=a4213d713b6f49d78d7bbb9484c9312f, scope.project.id=fcb9e5d02067447099445d3a4c0e470b, user.id=b41a9018f7564c729ffdb8838010138b) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:37:14.259968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351234c-5461-4df2-88a3-2d6305eb4a8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:14.261602 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5351234c-5461-4df2-88a3-2d6305eb4a8a None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:14.267459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 559/1130] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:37:14 2022] GET /identity/v3/role_assignments?user.id=b41a9018f7564c729ffdb8838010138b&scope.project.id=fcb9e5d02067447099445d3a4c0e470b&role.id=a4213d713b6f49d78d7bbb9484c9312f => generated 610 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:16.308100 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465cb2e-3a92-44c1-990c-cf54531ffd0e None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:16.308629 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465cb2e-3a92-44c1-990c-cf54531ffd0e None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:16.308629 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5465cb2e-3a92-44c1-990c-cf54531ffd0e None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:16.310003 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 572/1131] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:37:16 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:37:16.315339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a9a72-67fb-4d5c-8484-cfea73362dcf None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:16.315743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a9a72-67fb-4d5c-8484-cfea73362dcf None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:16.315888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a9a72-67fb-4d5c-8484-cfea73362dcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:16.334579 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-0d9a9a72-67fb-4d5c-8484-cfea73362dcf None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:37:16.335777 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-0d9a9a72-67fb-4d5c-8484-cfea73362dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:37:16.400720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 560/1132] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:37:16 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:37:16.424634 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465cb2e-3a92-44c1-990c-cf54531ffd0e None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:16.469588 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:16.527310 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5465cb2e-3a92-44c1-990c-cf54531ffd0e None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:16.561190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3965f8f-f27c-4495-910f-d5d6f917d283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:16.561736 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3965f8f-f27c-4495-910f-d5d6f917d283 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:16.562014 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3965f8f-f27c-4495-910f-d5d6f917d283 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:16.562283 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3965f8f-f27c-4495-910f-d5d6f917d283 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:16.563439 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3965f8f-f27c-4495-910f-d5d6f917d283 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:16.564859 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3965f8f-f27c-4495-910f-d5d6f917d283 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:16.566429 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 573/1133] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:37:16 2022] GET /identity/v3/domains/default => generated 200 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:16.572375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a9a72-67fb-4d5c-8484-cfea73362dcf None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:16.602459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:16.670741 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9a9a72-67fb-4d5c-8484-cfea73362dcf None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:16.703715 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f7fda5-e2ee-48fa-873a-01032f5d1607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:16.704316 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7fda5-e2ee-48fa-873a-01032f5d1607 None admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:16.704592 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7fda5-e2ee-48fa-873a-01032f5d1607 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:16.704861 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7fda5-e2ee-48fa-873a-01032f5d1607 None admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:16.705537 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f7fda5-e2ee-48fa-873a-01032f5d1607 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:16.706947 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f7fda5-e2ee-48fa-873a-01032f5d1607 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:16.797485 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 561/1134] 10.222.0.115 () {60 vars in 1146 bytes} [Tue Aug 2 10:37:16 2022] POST /identity/v3/projects => generated 312 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:37:18.818635 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feecc415-d01d-4991-8994-908b9e71e476 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:18.819198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feecc415-d01d-4991-8994-908b9e71e476 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:18.819281 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feecc415-d01d-4991-8994-908b9e71e476 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:18.820578 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 574/1135] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:37:18 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:37:18.826477 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c8c9b-6c01-4288-8a84-a76080efba2c None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:18.826918 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c8c9b-6c01-4288-8a84-a76080efba2c None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:18.827071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863c8c9b-6c01-4288-8a84-a76080efba2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:18.846239 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-863c8c9b-6c01-4288-8a84-a76080efba2c None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:37:18.847542 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-863c8c9b-6c01-4288-8a84-a76080efba2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:37:18.915041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 562/1136] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:37:18 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:37:18.940908 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feecc415-d01d-4991-8994-908b9e71e476 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:19.059338 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feecc415-d01d-4991-8994-908b9e71e476 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:19.068553 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:19.093502 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca063a9e-5067-4674-9c7f-cfb0d46622f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:19.094061 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca063a9e-5067-4674-9c7f-cfb0d46622f0 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:19.094339 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca063a9e-5067-4674-9c7f-cfb0d46622f0 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:19.094611 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca063a9e-5067-4674-9c7f-cfb0d46622f0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:19.095761 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca063a9e-5067-4674-9c7f-cfb0d46622f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:19.097176 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca063a9e-5067-4674-9c7f-cfb0d46622f0 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:19.099169 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 575/1137] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:37:18 2022] GET /identity/v3/domains/default => generated 200 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:19.105021 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c8c9b-6c01-4288-8a84-a76080efba2c None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:19.118848 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:19.204794 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863c8c9b-6c01-4288-8a84-a76080efba2c None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:19.238021 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038c9011-947e-4c1a-9590-56ecf0d785db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:19.238621 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c9011-947e-4c1a-9590-56ecf0d785db None admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:19.238896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c9011-947e-4c1a-9590-56ecf0d785db None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:19.239196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038c9011-947e-4c1a-9590-56ecf0d785db None admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:19.239864 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038c9011-947e-4c1a-9590-56ecf0d785db None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:19.241302 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038c9011-947e-4c1a-9590-56ecf0d785db None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:19.304086 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 563/1138] 10.222.0.115 () {60 vars in 1137 bytes} [Tue Aug 2 10:37:19 2022] POST /identity/v3/users => generated 287 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:37:21.368203 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca6618-87d0-4321-891d-f1cc053d824d None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:21.368805 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca6618-87d0-4321-891d-f1cc053d824d None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:21.368805 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca6618-87d0-4321-891d-f1cc053d824d None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:21.370216 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 576/1139] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:37:21 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:37:21.375702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc5add-640c-4b27-8a89-8e265e60d17b None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:21.376208 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc5add-640c-4b27-8a89-8e265e60d17b None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:21.376310 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbc5add-640c-4b27-8a89-8e265e60d17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:21.395362 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-ecbc5add-640c-4b27-8a89-8e265e60d17b None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:37:21.396548 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-ecbc5add-640c-4b27-8a89-8e265e60d17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:37:21.458533 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:21.464792 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 564/1140] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:37:21 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:37:21.488496 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca6618-87d0-4321-891d-f1cc053d824d None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:21.589043 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca6618-87d0-4321-891d-f1cc053d824d None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:21.621773 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1bbf2-21c3-4bac-a798-339608f55dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:21.622327 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1bbf2-21c3-4bac-a798-339608f55dd6 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:21.622602 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1bbf2-21c3-4bac-a798-339608f55dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:21.622870 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc1bbf2-21c3-4bac-a798-339608f55dd6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:21.632733 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc1bbf2-21c3-4bac-a798-339608f55dd6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:37:21.638405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-1bc1bbf2-21c3-4bac-a798-339608f55dd6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:37:21.638405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:37:21.638405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:37:21.638405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:37:21.638405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:37:21.638405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:37:21.638405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:37:21.638405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:37:21.638405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:37:21.638405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:37:21.638405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:37:21.638405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:37:21.638405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:37:21.638405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:37:21.638405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:37:21.638405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:37:21.639483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:37:21.639483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:37:21.639483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:37:21.639483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:37:21.639483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:37:21.639483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:37:21.639483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:37:21.639483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:37:21.639483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:37:21.639483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:37:21.639483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:37:21.639483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:37:21.639483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:37:21.639483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:37:21.639483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:37:21.639483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:37:21.640557 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:37:21.640557 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:37:21.640557 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:37:21.640557 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:37:21.640557 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:37:21.640557 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:37:21.640557 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:37:21.640557 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:37:21.640557 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:37:21.640557 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:37:21.640557 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:37:21.640557 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:37:21.640557 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 577/1141] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:37:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:37:21.645723 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc5add-640c-4b27-8a89-8e265e60d17b None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:21.742280 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbc5add-640c-4b27-8a89-8e265e60d17b None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:21.775426 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8af2d-a8df-40b3-8edd-9537d7c0fb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:21.776752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8af2d-a8df-40b3-8edd-9537d7c0fb3c None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:21.777206 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8af2d-a8df-40b3-8edd-9537d7c0fb3c None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:21.777498 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8af2d-a8df-40b3-8edd-9537d7c0fb3c None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:21.778214 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a8af2d-a8df-40b3-8edd-9537d7c0fb3c None admin] RBAC: Adding query filter params (name=admin) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:37:21.778521 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a8af2d-a8df-40b3-8edd-9537d7c0fb3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:21.779919 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a8af2d-a8df-40b3-8edd-9537d7c0fb3c None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:21.788136 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 565/1142] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:37:21 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:21.794224 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1bbf2-21c3-4bac-a798-339608f55dd6 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:21.829365 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:21.895227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc1bbf2-21c3-4bac-a798-339608f55dd6 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:21.928222 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520adaf5-00d9-41ef-a2b5-0ba5af83c9a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:21.928847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520adaf5-00d9-41ef-a2b5-0ba5af83c9a1 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:21.929128 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520adaf5-00d9-41ef-a2b5-0ba5af83c9a1 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:21.929402 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520adaf5-00d9-41ef-a2b5-0ba5af83c9a1 None admin] PATH_INFO: `/v3/users/2ddf59ae49ff4c2d9c13815fcf08a946` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:21.942911 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520adaf5-00d9-41ef-a2b5-0ba5af83c9a1 None admin] RBAC: Authorizing `identity:get_user(user_id=2ddf59ae49ff4c2d9c13815fcf08a946)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:21.944362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520adaf5-00d9-41ef-a2b5-0ba5af83c9a1 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:21.945974 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 578/1143] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:37:21 2022] GET /identity/v3/users/2ddf59ae49ff4c2d9c13815fcf08a946 => generated 287 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:21.952376 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8af2d-a8df-40b3-8edd-9537d7c0fb3c None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:22.010592 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:22.054885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8af2d-a8df-40b3-8edd-9537d7c0fb3c None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:22.091244 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af23fb-c09d-4d34-ba09-0a9add1c6fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:22.091866 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af23fb-c09d-4d34-ba09-0a9add1c6fb1 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:22.092144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af23fb-c09d-4d34-ba09-0a9add1c6fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:22.092414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af23fb-c09d-4d34-ba09-0a9add1c6fb1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:22.161208 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af23fb-c09d-4d34-ba09-0a9add1c6fb1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:22.162579 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af23fb-c09d-4d34-ba09-0a9add1c6fb1 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:22.224664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:22.269379 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 566/1144] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:37:21 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:37:22.275526 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520adaf5-00d9-41ef-a2b5-0ba5af83c9a1 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:22.372691 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:22.379939 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520adaf5-00d9-41ef-a2b5-0ba5af83c9a1 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:22.418522 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ef595-5ae2-4666-8617-4af439ce0ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:22.419103 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ef595-5ae2-4666-8617-4af439ce0ff0 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:22.419392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ef595-5ae2-4666-8617-4af439ce0ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:22.419672 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ef595-5ae2-4666-8617-4af439ce0ff0 None admin] PATH_INFO: `/v3/projects/917ea5ef757d43099d2decf9ab935cfb` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:22.420868 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ef595-5ae2-4666-8617-4af439ce0ff0 None admin] RBAC: Authorizing `identity:get_project(project_id=917ea5ef757d43099d2decf9ab935cfb)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:22.422306 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018ef595-5ae2-4666-8617-4af439ce0ff0 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:22.424244 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 579/1145] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:37:22 2022] GET /identity/v3/projects/917ea5ef757d43099d2decf9ab935cfb => generated 312 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:22.430564 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af23fb-c09d-4d34-ba09-0a9add1c6fb1 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:22.534380 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af23fb-c09d-4d34-ba09-0a9add1c6fb1 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:22.570123 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad75586-cc5a-4a4e-a882-80af362aae58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:22.570736 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad75586-cc5a-4a4e-a882-80af362aae58 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:22.571009 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad75586-cc5a-4a4e-a882-80af362aae58 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:22.571317 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad75586-cc5a-4a4e-a882-80af362aae58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:22.572104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad75586-cc5a-4a4e-a882-80af362aae58 None admin] RBAC: Adding query filter params (role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac, scope.project.id=917ea5ef757d43099d2decf9ab935cfb, user.id=2ddf59ae49ff4c2d9c13815fcf08a946) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:37:22.572414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad75586-cc5a-4a4e-a882-80af362aae58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:22.574053 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad75586-cc5a-4a4e-a882-80af362aae58 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:22.580489 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 567/1146] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:37:22 2022] GET /identity/v3/role_assignments?user.id=2ddf59ae49ff4c2d9c13815fcf08a946&scope.project.id=917ea5ef757d43099d2decf9ab935cfb&role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 260 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:24.572048 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c144faf-306f-4c19-bae4-2ad1ecf0e988 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:24.572610 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c144faf-306f-4c19-bae4-2ad1ecf0e988 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:24.572839 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c144faf-306f-4c19-bae4-2ad1ecf0e988 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:24.574292 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 580/1147] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:37:24 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:37:24.580092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966d9ec-3327-4e13-9120-3cff08c6310a None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:24.580552 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966d9ec-3327-4e13-9120-3cff08c6310a None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:24.580658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9966d9ec-3327-4e13-9120-3cff08c6310a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:24.598644 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-9966d9ec-3327-4e13-9120-3cff08c6310a None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:37:24.599870 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-9966d9ec-3327-4e13-9120-3cff08c6310a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:37:24.617091 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:24.664188 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 568/1148] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:37:24 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:37:24.687667 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c144faf-306f-4c19-bae4-2ad1ecf0e988 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:24.790237 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c144faf-306f-4c19-bae4-2ad1ecf0e988 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:24.825634 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0a3da-85e6-4725-92b4-c2c8daff118a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:24.826260 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0a3da-85e6-4725-92b4-c2c8daff118a None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:24.826536 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0a3da-85e6-4725-92b4-c2c8daff118a None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:24.826808 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc0a3da-85e6-4725-92b4-c2c8daff118a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:24.836815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc0a3da-85e6-4725-92b4-c2c8daff118a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:37:24.842304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-bdc0a3da-85e6-4725-92b4-c2c8daff118a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:37:24.842304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:37:24.842304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:37:24.842304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:37:24.842304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:37:24.842304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:37:24.842304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:37:24.842304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:37:24.842304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:37:24.842304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:37:24.842304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:37:24.842304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:37:24.842304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:37:24.842304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:37:24.842304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:37:24.842304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:37:24.843478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:37:24.843478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:37:24.843478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:37:24.843478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:37:24.843478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:37:24.843478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:37:24.843478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:37:24.843478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:37:24.843478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:37:24.843478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:37:24.843478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:37:24.843478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:37:24.843478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:37:24.843478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:37:24.843478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:37:24.843478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:37:24.844803 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:37:24.844803 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:37:24.844803 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:37:24.844803 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:37:24.844803 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:37:24.844803 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:37:24.844803 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:37:24.844803 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:37:24.844803 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:37:24.844803 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:37:24.844803 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:37:24.844803 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:37:24.844803 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 581/1149] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:37:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:37:24.849563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966d9ec-3327-4e13-9120-3cff08c6310a None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:24.953457 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9966d9ec-3327-4e13-9120-3cff08c6310a None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:24.990002 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c676eb-dd4c-4b24-bfd8-8027ea554d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:24.991015 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c676eb-dd4c-4b24-bfd8-8027ea554d5a None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:24.991015 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c676eb-dd4c-4b24-bfd8-8027ea554d5a None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:24.991257 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c676eb-dd4c-4b24-bfd8-8027ea554d5a None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:24.991916 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c676eb-dd4c-4b24-bfd8-8027ea554d5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:37:24.992217 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c676eb-dd4c-4b24-bfd8-8027ea554d5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:24.993605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c676eb-dd4c-4b24-bfd8-8027ea554d5a None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:25.002888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 569/1150] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:37:24 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:25.008809 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0a3da-85e6-4725-92b4-c2c8daff118a None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:25.035326 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:25.112442 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc0a3da-85e6-4725-92b4-c2c8daff118a None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:25.147695 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f98d2-4cc1-4d68-a1e0-36a2d8240428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:25.148337 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f98d2-4cc1-4d68-a1e0-36a2d8240428 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:25.148632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f98d2-4cc1-4d68-a1e0-36a2d8240428 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:25.148880 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210f98d2-4cc1-4d68-a1e0-36a2d8240428 None admin] PATH_INFO: `/v3/users/2ddf59ae49ff4c2d9c13815fcf08a946` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:25.150068 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210f98d2-4cc1-4d68-a1e0-36a2d8240428 None admin] RBAC: Authorizing `identity:get_user(user_id=2ddf59ae49ff4c2d9c13815fcf08a946)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:25.152186 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210f98d2-4cc1-4d68-a1e0-36a2d8240428 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:25.153805 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 582/1151] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:37:25 2022] GET /identity/v3/users/2ddf59ae49ff4c2d9c13815fcf08a946 => generated 287 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:25.160082 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c676eb-dd4c-4b24-bfd8-8027ea554d5a None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:25.179197 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:25.261171 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c676eb-dd4c-4b24-bfd8-8027ea554d5a None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:25.295247 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb0ded-7926-4bce-923b-a2708bf4b0ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:25.295887 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb0ded-7926-4bce-923b-a2708bf4b0ab None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:25.296168 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb0ded-7926-4bce-923b-a2708bf4b0ab None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:25.296444 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb0ded-7926-4bce-923b-a2708bf4b0ab None admin] PATH_INFO: `/v3/projects/917ea5ef757d43099d2decf9ab935cfb` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:25.297592 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccb0ded-7926-4bce-923b-a2708bf4b0ab None admin] RBAC: Authorizing `identity:get_project(project_id=917ea5ef757d43099d2decf9ab935cfb)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:25.299026 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccb0ded-7926-4bce-923b-a2708bf4b0ab None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:25.300723 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 570/1152] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:37:25 2022] GET /identity/v3/projects/917ea5ef757d43099d2decf9ab935cfb => generated 312 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:25.306998 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f98d2-4cc1-4d68-a1e0-36a2d8240428 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:25.395303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:25.405331 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210f98d2-4cc1-4d68-a1e0-36a2d8240428 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:25.439795 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbf22a7-6dd3-4e1b-b02d-d60612eb5a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:25.440455 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf22a7-6dd3-4e1b-b02d-d60612eb5a2e None admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:25.440735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf22a7-6dd3-4e1b-b02d-d60612eb5a2e None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:25.441011 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbf22a7-6dd3-4e1b-b02d-d60612eb5a2e None admin] PATH_INFO: `/v3/projects/917ea5ef757d43099d2decf9ab935cfb/users/2ddf59ae49ff4c2d9c13815fcf08a946/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:25.442918 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf22a7-6dd3-4e1b-b02d-d60612eb5a2e None admin] RBAC: Authorizing `identity:create_grant(project_id=917ea5ef757d43099d2decf9ab935cfb, user_id=2ddf59ae49ff4c2d9c13815fcf08a946, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:25.444376 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbf22a7-6dd3-4e1b-b02d-d60612eb5a2e None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:25.528197 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 583/1153] 10.222.0.115 () {58 vars in 1444 bytes} [Tue Aug 2 10:37:25 2022] PUT /identity/v3/projects/917ea5ef757d43099d2decf9ab935cfb/users/2ddf59ae49ff4c2d9c13815fcf08a946/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:37:27.521158 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426840e-ec79-4077-81f6-a7f62d158f0d None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:27.521699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426840e-ec79-4077-81f6-a7f62d158f0d None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:27.521699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426840e-ec79-4077-81f6-a7f62d158f0d None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:27.523064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 571/1154] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:37:27 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:37:27.529129 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db94fd4-85d7-4b49-b398-7b2aaa29c29a None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:27.529522 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db94fd4-85d7-4b49-b398-7b2aaa29c29a None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:27.529677 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db94fd4-85d7-4b49-b398-7b2aaa29c29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:27.547948 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-7db94fd4-85d7-4b49-b398-7b2aaa29c29a None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:37:27.549116 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-7db94fd4-85d7-4b49-b398-7b2aaa29c29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:37:27.613592 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 584/1155] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:37:27 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:37:27.637700 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5426840e-ec79-4077-81f6-a7f62d158f0d None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:27.720956 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:27.736646 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5426840e-ec79-4077-81f6-a7f62d158f0d None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:27.770702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f02ef-204e-4000-a0d3-ecca36ed10c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:27.771367 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f02ef-204e-4000-a0d3-ecca36ed10c7 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:27.771643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f02ef-204e-4000-a0d3-ecca36ed10c7 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:27.771913 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f02ef-204e-4000-a0d3-ecca36ed10c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:27.782539 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4f02ef-204e-4000-a0d3-ecca36ed10c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:37:27.787438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-8f4f02ef-204e-4000-a0d3-ecca36ed10c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:37:27.787438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:37:27.787438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:37:27.787438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:37:27.787438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:37:27.787438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:37:27.787438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:37:27.787438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:37:27.787438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:37:27.787438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:37:27.787438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:37:27.787438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:37:27.787438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:37:27.787438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:37:27.787438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:37:27.787438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:37:27.788601 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:37:27.788601 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:37:27.788601 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:37:27.788601 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:37:27.788601 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:37:27.788601 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:37:27.788601 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:37:27.788601 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:37:27.788601 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:37:27.788601 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:37:27.788601 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:37:27.788601 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:37:27.788601 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:37:27.788601 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:37:27.788601 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:37:27.788601 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:37:27.789843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:37:27.789843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:37:27.789843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:37:27.789843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:37:27.789843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:37:27.789843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:37:27.789843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:37:27.789843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:37:27.789843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:37:27.789843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:37:27.789843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:37:27.789843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:37:27.789843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 572/1156] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:37:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:37:27.794902 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db94fd4-85d7-4b49-b398-7b2aaa29c29a None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:27.876404 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:27.894903 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db94fd4-85d7-4b49-b398-7b2aaa29c29a None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:27.927221 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aace769-af57-4a85-b8a2-93cba32cb3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:27.927832 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aace769-af57-4a85-b8a2-93cba32cb3f2 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:27.928111 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aace769-af57-4a85-b8a2-93cba32cb3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:27.928383 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aace769-af57-4a85-b8a2-93cba32cb3f2 None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:27.929090 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aace769-af57-4a85-b8a2-93cba32cb3f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:37:27.929393 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aace769-af57-4a85-b8a2-93cba32cb3f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:27.930745 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aace769-af57-4a85-b8a2-93cba32cb3f2 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:27.939582 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 585/1157] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:37:27 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:27.945391 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f02ef-204e-4000-a0d3-ecca36ed10c7 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:28.043409 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f02ef-204e-4000-a0d3-ecca36ed10c7 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:28.077832 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616485e0-9c82-42ba-8547-4d33352bdf52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:28.078477 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616485e0-9c82-42ba-8547-4d33352bdf52 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:28.078755 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616485e0-9c82-42ba-8547-4d33352bdf52 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:28.079029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616485e0-9c82-42ba-8547-4d33352bdf52 None admin] PATH_INFO: `/v3/users/2ddf59ae49ff4c2d9c13815fcf08a946` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:28.080219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616485e0-9c82-42ba-8547-4d33352bdf52 None admin] RBAC: Authorizing `identity:get_user(user_id=2ddf59ae49ff4c2d9c13815fcf08a946)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:28.081639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616485e0-9c82-42ba-8547-4d33352bdf52 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:28.083266 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 573/1158] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:37:27 2022] GET /identity/v3/users/2ddf59ae49ff4c2d9c13815fcf08a946 => generated 287 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:28.089587 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aace769-af57-4a85-b8a2-93cba32cb3f2 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:28.187238 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aace769-af57-4a85-b8a2-93cba32cb3f2 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:28.219780 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91db580a-09b7-4f6c-a7a6-945b940ec048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:28.220389 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db580a-09b7-4f6c-a7a6-945b940ec048 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:28.220669 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db580a-09b7-4f6c-a7a6-945b940ec048 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:28.220945 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91db580a-09b7-4f6c-a7a6-945b940ec048 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:28.232912 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:28.285797 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91db580a-09b7-4f6c-a7a6-945b940ec048 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:28.287269 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91db580a-09b7-4f6c-a7a6-945b940ec048 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:28.386444 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 586/1159] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:37:28 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:37:28.392844 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616485e0-9c82-42ba-8547-4d33352bdf52 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:28.407643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:28.493814 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616485e0-9c82-42ba-8547-4d33352bdf52 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:28.528263 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a64649-8243-40aa-aacc-4c9938cf9199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:28.533573 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a64649-8243-40aa-aacc-4c9938cf9199 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:28.533573 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a64649-8243-40aa-aacc-4c9938cf9199 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:28.533573 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a64649-8243-40aa-aacc-4c9938cf9199 None admin] PATH_INFO: `/v3/projects/917ea5ef757d43099d2decf9ab935cfb` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:28.533573 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a64649-8243-40aa-aacc-4c9938cf9199 None admin] RBAC: Authorizing `identity:get_project(project_id=917ea5ef757d43099d2decf9ab935cfb)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:28.533573 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a64649-8243-40aa-aacc-4c9938cf9199 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:28.534099 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 574/1160] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:37:28 2022] GET /identity/v3/projects/917ea5ef757d43099d2decf9ab935cfb => generated 312 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:28.539747 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91db580a-09b7-4f6c-a7a6-945b940ec048 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:28.581795 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:28.637594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91db580a-09b7-4f6c-a7a6-945b940ec048 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:28.669672 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606c60d0-47e5-4bfc-a6e2-d03180a8953f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:28.670272 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c60d0-47e5-4bfc-a6e2-d03180a8953f None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:28.670557 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c60d0-47e5-4bfc-a6e2-d03180a8953f None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:28.670838 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606c60d0-47e5-4bfc-a6e2-d03180a8953f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:28.671647 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606c60d0-47e5-4bfc-a6e2-d03180a8953f None admin] RBAC: Adding query filter params (role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac, scope.project.id=917ea5ef757d43099d2decf9ab935cfb, user.id=2ddf59ae49ff4c2d9c13815fcf08a946) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:37:28.671980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606c60d0-47e5-4bfc-a6e2-d03180a8953f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:28.673404 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606c60d0-47e5-4bfc-a6e2-d03180a8953f None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:28.679028 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 587/1161] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:37:28 2022] GET /identity/v3/role_assignments?user.id=2ddf59ae49ff4c2d9c13815fcf08a946&scope.project.id=917ea5ef757d43099d2decf9ab935cfb&role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 610 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:30.682047 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba8742-1085-45c9-867e-9b447c3f7fdd None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:30.682628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba8742-1085-45c9-867e-9b447c3f7fdd None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:30.682628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba8742-1085-45c9-867e-9b447c3f7fdd None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:30.683951 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 575/1162] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:37:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:37:30.689109 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7e9d2-61f7-40f3-941a-b713a2d1206a None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:30.689514 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7e9d2-61f7-40f3-941a-b713a2d1206a None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:30.689657 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad7e9d2-61f7-40f3-941a-b713a2d1206a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:30.708395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-5ad7e9d2-61f7-40f3-941a-b713a2d1206a None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:37:30.709517 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-5ad7e9d2-61f7-40f3-941a-b713a2d1206a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:37:30.773918 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 588/1163] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:37:30 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:37:30.796755 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba8742-1085-45c9-867e-9b447c3f7fdd None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:30.875614 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:30.896049 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba8742-1085-45c9-867e-9b447c3f7fdd None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:30.930633 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b96a9-e107-4a75-847b-9ac26fd20459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:30.931266 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b96a9-e107-4a75-847b-9ac26fd20459 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:30.932064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b96a9-e107-4a75-847b-9ac26fd20459 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:30.932333 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b96a9-e107-4a75-847b-9ac26fd20459 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:31.000354 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b96a9-e107-4a75-847b-9ac26fd20459 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:31.001716 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b96a9-e107-4a75-847b-9ac26fd20459 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:31.083940 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:31.103413 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 576/1164] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:37:30 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:37:31.109644 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7e9d2-61f7-40f3-941a-b713a2d1206a None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:31.143970 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:31.207904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad7e9d2-61f7-40f3-941a-b713a2d1206a None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:31.241095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3a0f6-5be8-4b4b-a634-a39dede18f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:31.241719 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3a0f6-5be8-4b4b-a634-a39dede18f07 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:31.241999 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3a0f6-5be8-4b4b-a634-a39dede18f07 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:31.242272 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3a0f6-5be8-4b4b-a634-a39dede18f07 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:31.249570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c3a0f6-5be8-4b4b-a634-a39dede18f07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:31.251073 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c3a0f6-5be8-4b4b-a634-a39dede18f07 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:31.262371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-f4c3a0f6-5be8-4b4b-a634-a39dede18f07 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 02 10:37:31.262371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:37:31.262371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 02 10:37:31.262371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 02 10:37:31.262371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:37:31.262371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:37:31.262371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:37:31.262371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:37:31.262371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 02 10:37:31.262371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:37:31.262371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:37:31.262371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:37:31.262371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:37:31.262371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:37:31.262371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:37:31.262371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:37:31.262371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:37:31.263658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:37:31.263658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:37:31.263658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:37:31.263658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:37:31.263658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:37:31.263658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:37:31.263658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:37:31.263658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:37:31.263658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:37:31.263658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:37:31.263658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:37:31.263658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:37:31.263658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:37:31.263658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:37:31.263658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:37:31.263658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:37:31.264609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:37:31.264609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:37:31.264609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:37:31.264609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:37:31.264609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:37:31.264609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:37:31.264609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:37:31.264609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:37:31.264609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:37:31.264609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:37:31.264609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 02 10:37:31.264609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:37:31.264609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 02 10:37:31.264609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:37:31.264609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 589/1165] 10.222.0.115 () {56 vars in 1122 bytes} [Tue Aug 2 10:37:31 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:37:31.269564 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b96a9-e107-4a75-847b-9ac26fd20459 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:31.366596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b96a9-e107-4a75-847b-9ac26fd20459 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:31.400630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcfc89b-c50c-4934-b7d5-75f8d99e564a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:31.400814 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcfc89b-c50c-4934-b7d5-75f8d99e564a None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:31.401041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcfc89b-c50c-4934-b7d5-75f8d99e564a None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:31.401312 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcfc89b-c50c-4934-b7d5-75f8d99e564a None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:31.401990 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcfc89b-c50c-4934-b7d5-75f8d99e564a None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:37:31.402289 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcfc89b-c50c-4934-b7d5-75f8d99e564a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:31.403705 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcfc89b-c50c-4934-b7d5-75f8d99e564a None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:31.410668 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 577/1166] 10.222.0.115 () {56 vars in 1120 bytes} [Tue Aug 2 10:37:31 2022] GET /identity/v3/domains?name=swift_test => generated 126 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:33.406705 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e480785-f365-4e51-9e2f-24d7ec81b014 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:33.407299 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e480785-f365-4e51-9e2f-24d7ec81b014 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:33.407299 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e480785-f365-4e51-9e2f-24d7ec81b014 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:33.408635 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 590/1167] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:37:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:37:33.413866 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd693fc7-a42d-4db9-beb1-73cafceea418 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:33.414267 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd693fc7-a42d-4db9-beb1-73cafceea418 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:33.414422 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd693fc7-a42d-4db9-beb1-73cafceea418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:33.433023 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-bd693fc7-a42d-4db9-beb1-73cafceea418 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:37:33.434226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-bd693fc7-a42d-4db9-beb1-73cafceea418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:37:33.475279 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:33.501733 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 578/1168] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:37:33 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:37:33.524897 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e480785-f365-4e51-9e2f-24d7ec81b014 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:33.617702 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:33.624639 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e480785-f365-4e51-9e2f-24d7ec81b014 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:33.657905 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb06866-9aa6-43aa-b4b4-6dad606401ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:33.658434 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb06866-9aa6-43aa-b4b4-6dad606401ba None admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:33.658709 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb06866-9aa6-43aa-b4b4-6dad606401ba None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:33.658975 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb06866-9aa6-43aa-b4b4-6dad606401ba None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:33.659706 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb06866-9aa6-43aa-b4b4-6dad606401ba None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:33.661039 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb06866-9aa6-43aa-b4b4-6dad606401ba None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:33.739232 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 591/1169] 10.222.0.115 () {60 vars in 1143 bytes} [Tue Aug 2 10:37:33 2022] POST /identity/v3/domains => generated 268 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:37:35.751286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30693ee-8564-4c7a-b6a6-4954436c34d4 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:35.751795 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30693ee-8564-4c7a-b6a6-4954436c34d4 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:35.751795 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30693ee-8564-4c7a-b6a6-4954436c34d4 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:35.753016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 579/1170] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:37:35 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:37:35.758049 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b810590-ec19-4962-80e4-66cf8ca8e624 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:35.758439 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b810590-ec19-4962-80e4-66cf8ca8e624 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:35.758599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b810590-ec19-4962-80e4-66cf8ca8e624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:35.777403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-7b810590-ec19-4962-80e4-66cf8ca8e624 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:37:35.778567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-7b810590-ec19-4962-80e4-66cf8ca8e624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:37:35.844591 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 592/1171] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:37:35 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:37:35.867888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30693ee-8564-4c7a-b6a6-4954436c34d4 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:35.967478 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30693ee-8564-4c7a-b6a6-4954436c34d4 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:36.000004 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8879b6-0c97-4e41-8066-31ca9381c081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:36.000623 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8879b6-0c97-4e41-8066-31ca9381c081 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:36.000898 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8879b6-0c97-4e41-8066-31ca9381c081 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:36.001166 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8879b6-0c97-4e41-8066-31ca9381c081 None admin] PATH_INFO: `/v3/domains/5554e4ac0e904dceb6bbbc01f9889a20` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:36.011911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8879b6-0c97-4e41-8066-31ca9381c081 None admin] RBAC: Authorizing `identity:get_domain(domain_id=5554e4ac0e904dceb6bbbc01f9889a20)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:36.013283 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df8879b6-0c97-4e41-8066-31ca9381c081 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:36.014923 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 580/1172] 10.222.0.115 () {56 vars in 1188 bytes} [Tue Aug 2 10:37:35 2022] GET /identity/v3/domains/5554e4ac0e904dceb6bbbc01f9889a20 => generated 268 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:36.020782 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b810590-ec19-4962-80e4-66cf8ca8e624 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:36.102823 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:36.116851 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b810590-ec19-4962-80e4-66cf8ca8e624 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:36.151496 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ba933d-7367-4328-8343-fb9a1073bae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:36.152112 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba933d-7367-4328-8343-fb9a1073bae2 None admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:36.152394 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba933d-7367-4328-8343-fb9a1073bae2 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:36.152668 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba933d-7367-4328-8343-fb9a1073bae2 None admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:36.153344 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ba933d-7367-4328-8343-fb9a1073bae2 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:36.154800 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ba933d-7367-4328-8343-fb9a1073bae2 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:36.253405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 593/1173] 10.222.0.115 () {60 vars in 1146 bytes} [Tue Aug 2 10:37:36 2022] POST /identity/v3/projects => generated 362 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:37:38.276327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b00fd7-3ace-40cb-a6fd-36fc79397175 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:38.276887 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b00fd7-3ace-40cb-a6fd-36fc79397175 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:38.276887 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b00fd7-3ace-40cb-a6fd-36fc79397175 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:38.278226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 581/1174] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:37:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:37:38.283507 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5313b3-92b0-454a-9151-3da2d735301a None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:38.283940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5313b3-92b0-454a-9151-3da2d735301a None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:38.284075 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5313b3-92b0-454a-9151-3da2d735301a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:38.302703 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-3b5313b3-92b0-454a-9151-3da2d735301a None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:37:38.303949 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-3b5313b3-92b0-454a-9151-3da2d735301a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:37:38.371687 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 594/1175] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:37:38 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:37:38.396984 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b00fd7-3ace-40cb-a6fd-36fc79397175 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:38.432739 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:38.498280 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b00fd7-3ace-40cb-a6fd-36fc79397175 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:38.531155 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b9046-c8ca-4934-8d8b-5e3634c8c05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:38.531725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b9046-c8ca-4934-8d8b-5e3634c8c05e None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:38.532001 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b9046-c8ca-4934-8d8b-5e3634c8c05e None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:38.532281 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b9046-c8ca-4934-8d8b-5e3634c8c05e None admin] PATH_INFO: `/v3/domains/5554e4ac0e904dceb6bbbc01f9889a20` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:38.533424 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b9046-c8ca-4934-8d8b-5e3634c8c05e None admin] RBAC: Authorizing `identity:get_domain(domain_id=5554e4ac0e904dceb6bbbc01f9889a20)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:38.534847 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b9046-c8ca-4934-8d8b-5e3634c8c05e None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:38.536489 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 582/1176] 10.222.0.115 () {56 vars in 1188 bytes} [Tue Aug 2 10:37:38 2022] GET /identity/v3/domains/5554e4ac0e904dceb6bbbc01f9889a20 => generated 268 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:38.542670 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5313b3-92b0-454a-9151-3da2d735301a None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:38.607248 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:38.639841 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5313b3-92b0-454a-9151-3da2d735301a None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:38.672837 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c613dba-7f65-4f18-a9bd-8074c0e6c37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:38.673449 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c613dba-7f65-4f18-a9bd-8074c0e6c37a None admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:38.673736 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c613dba-7f65-4f18-a9bd-8074c0e6c37a None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:38.674018 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c613dba-7f65-4f18-a9bd-8074c0e6c37a None admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:38.674957 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c613dba-7f65-4f18-a9bd-8074c0e6c37a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:38.676676 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c613dba-7f65-4f18-a9bd-8074c0e6c37a None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:38.744549 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 595/1177] 10.222.0.115 () {60 vars in 1137 bytes} [Tue Aug 2 10:37:38 2022] POST /identity/v3/users => generated 312 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:37:40.785037 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a69977-7ac3-44ff-bdb8-affdd64a7001 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:40.785595 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a69977-7ac3-44ff-bdb8-affdd64a7001 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:40.785595 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a69977-7ac3-44ff-bdb8-affdd64a7001 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:40.786986 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 583/1178] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:37:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:37:40.792307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed181e-409d-4f66-8efc-0bde243313fe None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:40.792743 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed181e-409d-4f66-8efc-0bde243313fe None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:40.792856 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bed181e-409d-4f66-8efc-0bde243313fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:40.811248 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-4bed181e-409d-4f66-8efc-0bde243313fe None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:37:40.812443 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-4bed181e-409d-4f66-8efc-0bde243313fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:37:40.877357 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 596/1179] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:37:40 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:37:40.901244 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a69977-7ac3-44ff-bdb8-affdd64a7001 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:41.013146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a69977-7ac3-44ff-bdb8-affdd64a7001 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:41.014698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:41.050396 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808163f6-20a6-45bc-8ebb-37ff5ae80d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:41.050946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808163f6-20a6-45bc-8ebb-37ff5ae80d52 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:41.051250 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808163f6-20a6-45bc-8ebb-37ff5ae80d52 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:41.051518 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808163f6-20a6-45bc-8ebb-37ff5ae80d52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:41.062481 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808163f6-20a6-45bc-8ebb-37ff5ae80d52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:37:41.067402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-808163f6-20a6-45bc-8ebb-37ff5ae80d52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:37:41.067402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:37:41.067402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:37:41.067402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:37:41.067402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:37:41.067402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:37:41.067402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:37:41.067402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:37:41.067402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:37:41.067402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:37:41.067402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:37:41.067402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:37:41.067402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:37:41.067402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:37:41.067402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:37:41.067402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:37:41.068511 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:37:41.068511 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:37:41.068511 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:37:41.068511 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:37:41.068511 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:37:41.068511 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:37:41.068511 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:37:41.068511 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:37:41.068511 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:37:41.068511 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:37:41.068511 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:37:41.068511 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:37:41.068511 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:37:41.068511 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:37:41.068511 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:37:41.068511 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:37:41.069595 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:37:41.069595 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:37:41.069595 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:37:41.069595 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:37:41.069595 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:37:41.069595 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:37:41.069595 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:37:41.069595 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:37:41.069595 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:37:41.069595 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:37:41.069595 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:37:41.069595 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:37:41.069595 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 584/1180] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:37:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:37:41.075033 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed181e-409d-4f66-8efc-0bde243313fe None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:41.140531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:41.186244 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bed181e-409d-4f66-8efc-0bde243313fe None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:41.223726 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f4618-9cd8-414b-9fcf-d858b7199615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:41.224334 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f4618-9cd8-414b-9fcf-d858b7199615 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:41.224612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f4618-9cd8-414b-9fcf-d858b7199615 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:41.224911 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f4618-9cd8-414b-9fcf-d858b7199615 None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:41.225618 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f4618-9cd8-414b-9fcf-d858b7199615 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:37:41.225918 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f4618-9cd8-414b-9fcf-d858b7199615 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:41.227295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f4618-9cd8-414b-9fcf-d858b7199615 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:41.235946 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 597/1181] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:37:41 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:41.241703 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808163f6-20a6-45bc-8ebb-37ff5ae80d52 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:41.341809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808163f6-20a6-45bc-8ebb-37ff5ae80d52 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:41.375184 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409722d-9312-45f7-9af0-791efdae7ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:41.375812 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409722d-9312-45f7-9af0-791efdae7ef4 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:41.376090 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409722d-9312-45f7-9af0-791efdae7ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:41.376363 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409722d-9312-45f7-9af0-791efdae7ef4 None admin] PATH_INFO: `/v3/users/76dfbb65caa1403fbb797d0a8d4ce9cb` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:41.389459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409722d-9312-45f7-9af0-791efdae7ef4 None admin] RBAC: Authorizing `identity:get_user(user_id=76dfbb65caa1403fbb797d0a8d4ce9cb)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:41.390894 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409722d-9312-45f7-9af0-791efdae7ef4 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:41.392577 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 585/1182] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:37:41 2022] GET /identity/v3/users/76dfbb65caa1403fbb797d0a8d4ce9cb => generated 312 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:41.398336 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f4618-9cd8-414b-9fcf-d858b7199615 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:41.498748 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f4618-9cd8-414b-9fcf-d858b7199615 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:41.514462 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:41.534686 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca9166-b64e-4d0f-b202-b5cda4e49e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:41.535321 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca9166-b64e-4d0f-b202-b5cda4e49e2b None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:41.535601 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca9166-b64e-4d0f-b202-b5cda4e49e2b None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:41.535870 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ca9166-b64e-4d0f-b202-b5cda4e49e2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:41.651565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca9166-b64e-4d0f-b202-b5cda4e49e2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:41.652925 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ca9166-b64e-4d0f-b202-b5cda4e49e2b None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:41.752771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 598/1183] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:37:41 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:37:41.758668 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409722d-9312-45f7-9af0-791efdae7ef4 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:41.783824 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:41.865006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409722d-9312-45f7-9af0-791efdae7ef4 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:41.901813 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739d0a14-0a74-459a-b7a7-7f4bad384d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:41.902421 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d0a14-0a74-459a-b7a7-7f4bad384d71 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:41.902702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d0a14-0a74-459a-b7a7-7f4bad384d71 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:41.902978 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739d0a14-0a74-459a-b7a7-7f4bad384d71 None admin] PATH_INFO: `/v3/projects/8dcda29e027a4e6ab74fcbfb53e24c5f` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:41.904163 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739d0a14-0a74-459a-b7a7-7f4bad384d71 None admin] RBAC: Authorizing `identity:get_project(project_id=8dcda29e027a4e6ab74fcbfb53e24c5f)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:41.905615 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739d0a14-0a74-459a-b7a7-7f4bad384d71 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:41.907328 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 586/1184] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:37:41 2022] GET /identity/v3/projects/8dcda29e027a4e6ab74fcbfb53e24c5f => generated 362 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:41.913902 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca9166-b64e-4d0f-b202-b5cda4e49e2b None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:41.931225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:42.022040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ca9166-b64e-4d0f-b202-b5cda4e49e2b None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:42.057318 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6907807-fa5d-4209-9f7b-d0e8eaae6165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:42.057931 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6907807-fa5d-4209-9f7b-d0e8eaae6165 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:42.058214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6907807-fa5d-4209-9f7b-d0e8eaae6165 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:42.058487 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6907807-fa5d-4209-9f7b-d0e8eaae6165 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:42.059306 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6907807-fa5d-4209-9f7b-d0e8eaae6165 None admin] RBAC: Adding query filter params (role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac, scope.project.id=8dcda29e027a4e6ab74fcbfb53e24c5f, user.id=76dfbb65caa1403fbb797d0a8d4ce9cb) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:37:42.059618 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6907807-fa5d-4209-9f7b-d0e8eaae6165 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:42.061027 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6907807-fa5d-4209-9f7b-d0e8eaae6165 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:42.066537 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 599/1185] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:37:41 2022] GET /identity/v3/role_assignments?user.id=76dfbb65caa1403fbb797d0a8d4ce9cb&scope.project.id=8dcda29e027a4e6ab74fcbfb53e24c5f&role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 260 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:44.053903 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c99ca7-c1ef-4c44-a61c-dcd534114ca3 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:44.054451 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c99ca7-c1ef-4c44-a61c-dcd534114ca3 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:44.054451 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c99ca7-c1ef-4c44-a61c-dcd534114ca3 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:44.055870 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 587/1186] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:37:44 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:37:44.062195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b49ca9-d637-4f19-bdaf-f37ad47026c2 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:44.062717 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b49ca9-d637-4f19-bdaf-f37ad47026c2 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:44.062717 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b49ca9-d637-4f19-bdaf-f37ad47026c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:44.081365 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-79b49ca9-d637-4f19-bdaf-f37ad47026c2 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:37:44.082544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-79b49ca9-d637-4f19-bdaf-f37ad47026c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:37:44.148309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 600/1187] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:37:44 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:37:44.171313 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c99ca7-c1ef-4c44-a61c-dcd534114ca3 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:44.258219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:44.271342 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c99ca7-c1ef-4c44-a61c-dcd534114ca3 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:44.304866 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0346f6-f1e1-4a6b-99f1-4a5ef4902cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:44.305423 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0346f6-f1e1-4a6b-99f1-4a5ef4902cad None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:44.305703 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0346f6-f1e1-4a6b-99f1-4a5ef4902cad None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:44.306550 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0346f6-f1e1-4a6b-99f1-4a5ef4902cad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:44.316548 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0346f6-f1e1-4a6b-99f1-4a5ef4902cad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:37:44.321467 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-1c0346f6-f1e1-4a6b-99f1-4a5ef4902cad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:37:44.321467 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:37:44.321467 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:37:44.321467 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:37:44.321467 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:37:44.321467 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:37:44.321467 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:37:44.321467 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:37:44.321467 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:37:44.321467 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:37:44.321467 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:37:44.321467 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:37:44.321467 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:37:44.321467 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:37:44.321467 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:37:44.321467 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:37:44.322721 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:37:44.322721 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:37:44.322721 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:37:44.322721 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:37:44.322721 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:37:44.322721 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:37:44.322721 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:37:44.322721 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:37:44.322721 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:37:44.322721 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:37:44.322721 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:37:44.322721 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:37:44.322721 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:37:44.322721 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:37:44.322721 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:37:44.322721 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:37:44.323869 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:37:44.323869 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:37:44.323869 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:37:44.323869 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:37:44.323869 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:37:44.323869 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:37:44.323869 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:37:44.323869 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:37:44.323869 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:37:44.323869 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:37:44.323869 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:37:44.323869 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:37:44.323869 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 588/1188] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:37:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:37:44.329146 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b49ca9-d637-4f19-bdaf-f37ad47026c2 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:44.341116 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:44.427061 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b49ca9-d637-4f19-bdaf-f37ad47026c2 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:44.460934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba3b22-7515-4dda-8453-f299a0fadf5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:44.461555 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba3b22-7515-4dda-8453-f299a0fadf5a None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:44.461844 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba3b22-7515-4dda-8453-f299a0fadf5a None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:44.462762 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ba3b22-7515-4dda-8453-f299a0fadf5a None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:44.463524 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ba3b22-7515-4dda-8453-f299a0fadf5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:37:44.463831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ba3b22-7515-4dda-8453-f299a0fadf5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:44.465191 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ba3b22-7515-4dda-8453-f299a0fadf5a None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:44.473315 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 601/1189] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:37:44 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:44.479797 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0346f6-f1e1-4a6b-99f1-4a5ef4902cad None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:44.586138 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0346f6-f1e1-4a6b-99f1-4a5ef4902cad None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:44.621122 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010d5de-5d47-4330-8193-3df7e4e087b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:44.621755 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010d5de-5d47-4330-8193-3df7e4e087b3 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:44.622057 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010d5de-5d47-4330-8193-3df7e4e087b3 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:44.622335 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010d5de-5d47-4330-8193-3df7e4e087b3 None admin] PATH_INFO: `/v3/users/76dfbb65caa1403fbb797d0a8d4ce9cb` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:44.623522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4010d5de-5d47-4330-8193-3df7e4e087b3 None admin] RBAC: Authorizing `identity:get_user(user_id=76dfbb65caa1403fbb797d0a8d4ce9cb)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:44.624950 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4010d5de-5d47-4330-8193-3df7e4e087b3 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:44.626564 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 589/1190] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:37:44 2022] GET /identity/v3/users/76dfbb65caa1403fbb797d0a8d4ce9cb => generated 312 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:37:44.632340 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba3b22-7515-4dda-8453-f299a0fadf5a None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:44.701128 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:44.731853 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ba3b22-7515-4dda-8453-f299a0fadf5a None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:44.766028 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39f296-a9f8-47bc-bf36-9a1d16f5a3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:44.766660 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39f296-a9f8-47bc-bf36-9a1d16f5a3c0 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:44.767029 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39f296-a9f8-47bc-bf36-9a1d16f5a3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:44.767242 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39f296-a9f8-47bc-bf36-9a1d16f5a3c0 None admin] PATH_INFO: `/v3/projects/8dcda29e027a4e6ab74fcbfb53e24c5f` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:44.769062 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39f296-a9f8-47bc-bf36-9a1d16f5a3c0 None admin] RBAC: Authorizing `identity:get_project(project_id=8dcda29e027a4e6ab74fcbfb53e24c5f)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:44.769789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39f296-a9f8-47bc-bf36-9a1d16f5a3c0 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:44.771504 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 602/1191] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:37:44 2022] GET /identity/v3/projects/8dcda29e027a4e6ab74fcbfb53e24c5f => generated 362 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:44.778166 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010d5de-5d47-4330-8193-3df7e4e087b3 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:44.799611 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:44.882915 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010d5de-5d47-4330-8193-3df7e4e087b3 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:44.917148 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7761f-45b2-46ad-8c8a-e7cfad7918ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:44.918000 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7761f-45b2-46ad-8c8a-e7cfad7918ea None admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:44.918378 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7761f-45b2-46ad-8c8a-e7cfad7918ea None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:44.918757 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7761f-45b2-46ad-8c8a-e7cfad7918ea None admin] PATH_INFO: `/v3/projects/8dcda29e027a4e6ab74fcbfb53e24c5f/users/76dfbb65caa1403fbb797d0a8d4ce9cb/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:44.921227 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e7761f-45b2-46ad-8c8a-e7cfad7918ea None admin] RBAC: Authorizing `identity:create_grant(project_id=8dcda29e027a4e6ab74fcbfb53e24c5f, user_id=76dfbb65caa1403fbb797d0a8d4ce9cb, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:44.922663 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e7761f-45b2-46ad-8c8a-e7cfad7918ea None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:44.984244 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 590/1192] 10.222.0.115 () {58 vars in 1444 bytes} [Tue Aug 2 10:37:44 2022] PUT /identity/v3/projects/8dcda29e027a4e6ab74fcbfb53e24c5f/users/76dfbb65caa1403fbb797d0a8d4ce9cb/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:37:47.002986 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483d6e1-b3ff-4b38-b32f-15a97ca0e176 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:47.003563 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483d6e1-b3ff-4b38-b32f-15a97ca0e176 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:47.003683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483d6e1-b3ff-4b38-b32f-15a97ca0e176 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:47.004943 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 603/1193] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:37:47 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:37:47.015935 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9faa4c-3879-44c2-9070-7e609fdcd2d0 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:47.016404 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9faa4c-3879-44c2-9070-7e609fdcd2d0 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:47.016515 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9faa4c-3879-44c2-9070-7e609fdcd2d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:47.034245 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-ca9faa4c-3879-44c2-9070-7e609fdcd2d0 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:37:47.035480 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-ca9faa4c-3879-44c2-9070-7e609fdcd2d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:37:47.108058 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 591/1194] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:37:47 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:37:47.131791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483d6e1-b3ff-4b38-b32f-15a97ca0e176 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:47.241185 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483d6e1-b3ff-4b38-b32f-15a97ca0e176 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:47.276377 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5fe0e7-5789-43e0-89d0-0b1914b9e507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:47.277018 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5fe0e7-5789-43e0-89d0-0b1914b9e507 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:47.277316 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5fe0e7-5789-43e0-89d0-0b1914b9e507 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:47.277576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5fe0e7-5789-43e0-89d0-0b1914b9e507 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:47.285123 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:47.288148 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5fe0e7-5789-43e0-89d0-0b1914b9e507 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:37:47.293705 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-0e5fe0e7-5789-43e0-89d0-0b1914b9e507 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:37:47.293705 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:37:47.293705 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:37:47.293705 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:37:47.293705 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:37:47.293705 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:37:47.293705 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:37:47.293705 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:37:47.293705 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:37:47.293705 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:37:47.293705 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:37:47.293705 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:37:47.293705 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:37:47.293705 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:37:47.293705 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:37:47.293705 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:37:47.301875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:37:47.301875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:37:47.301875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:37:47.301875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:37:47.301875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:37:47.301875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:37:47.301875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:37:47.301875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:37:47.301875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:37:47.301875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:37:47.301875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:37:47.301875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:37:47.301875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:37:47.301875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:37:47.301875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:37:47.301875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:37:47.302979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:37:47.302979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:37:47.302979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:37:47.302979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:37:47.302979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:37:47.302979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:37:47.302979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:37:47.302979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:37:47.302979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:37:47.302979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:37:47.302979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:37:47.302979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:37:47.302979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 604/1195] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:37:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:37:47.303978 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9faa4c-3879-44c2-9070-7e609fdcd2d0 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:47.310723 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:47.400678 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9faa4c-3879-44c2-9070-7e609fdcd2d0 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:47.433756 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d47f42-3884-4a10-8cca-6f74d1c77f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:47.434350 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d47f42-3884-4a10-8cca-6f74d1c77f8e None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:47.434623 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d47f42-3884-4a10-8cca-6f74d1c77f8e None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:47.434889 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d47f42-3884-4a10-8cca-6f74d1c77f8e None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:47.435623 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d47f42-3884-4a10-8cca-6f74d1c77f8e None admin] RBAC: Adding query filter params (name=admin) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:37:47.435922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d47f42-3884-4a10-8cca-6f74d1c77f8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:47.437319 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d47f42-3884-4a10-8cca-6f74d1c77f8e None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:47.445574 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 592/1196] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:37:47 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:47.451523 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5fe0e7-5789-43e0-89d0-0b1914b9e507 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:47.553230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5fe0e7-5789-43e0-89d0-0b1914b9e507 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:47.586747 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2355f-5411-4495-a72c-9e40a680f859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:47.588009 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2355f-5411-4495-a72c-9e40a680f859 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:47.588290 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2355f-5411-4495-a72c-9e40a680f859 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:47.588566 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc2355f-5411-4495-a72c-9e40a680f859 None admin] PATH_INFO: `/v3/users/76dfbb65caa1403fbb797d0a8d4ce9cb` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:47.589734 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc2355f-5411-4495-a72c-9e40a680f859 None admin] RBAC: Authorizing `identity:get_user(user_id=76dfbb65caa1403fbb797d0a8d4ce9cb)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:47.591216 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc2355f-5411-4495-a72c-9e40a680f859 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:47.592817 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 605/1197] 10.222.0.115 () {56 vars in 1182 bytes} [Tue Aug 2 10:37:47 2022] GET /identity/v3/users/76dfbb65caa1403fbb797d0a8d4ce9cb => generated 312 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:47.598677 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d47f42-3884-4a10-8cca-6f74d1c77f8e None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:47.664770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:47.697923 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d47f42-3884-4a10-8cca-6f74d1c77f8e None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:47.731805 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313fa2d-d7e6-42a5-9cf8-409525314ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:47.732412 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313fa2d-d7e6-42a5-9cf8-409525314ad1 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:47.732694 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313fa2d-d7e6-42a5-9cf8-409525314ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:47.732970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0313fa2d-d7e6-42a5-9cf8-409525314ad1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:47.798984 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0313fa2d-d7e6-42a5-9cf8-409525314ad1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:47.800522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0313fa2d-d7e6-42a5-9cf8-409525314ad1 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:47.868988 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:47.905457 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 593/1198] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:37:47 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 10:37:47.911451 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2355f-5411-4495-a72c-9e40a680f859 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:47.978721 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:48.011296 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc2355f-5411-4495-a72c-9e40a680f859 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:48.044149 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9439b1-6a31-4f85-bc60-5cd95faf5a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:48.044903 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9439b1-6a31-4f85-bc60-5cd95faf5a10 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:48.045179 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9439b1-6a31-4f85-bc60-5cd95faf5a10 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:48.045452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9439b1-6a31-4f85-bc60-5cd95faf5a10 None admin] PATH_INFO: `/v3/projects/8dcda29e027a4e6ab74fcbfb53e24c5f` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:48.046577 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9439b1-6a31-4f85-bc60-5cd95faf5a10 None admin] RBAC: Authorizing `identity:get_project(project_id=8dcda29e027a4e6ab74fcbfb53e24c5f)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:48.048083 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9439b1-6a31-4f85-bc60-5cd95faf5a10 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:48.049734 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 606/1199] 10.222.0.115 () {56 vars in 1191 bytes} [Tue Aug 2 10:37:47 2022] GET /identity/v3/projects/8dcda29e027a4e6ab74fcbfb53e24c5f => generated 362 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:48.056572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313fa2d-d7e6-42a5-9cf8-409525314ad1 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:37:48.152116 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0313fa2d-d7e6-42a5-9cf8-409525314ad1 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:37:48.185145 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b93965-f384-49cc-9a91-2f0ea9a12fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:37:48.185741 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b93965-f384-49cc-9a91-2f0ea9a12fe2 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:37:48.186017 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b93965-f384-49cc-9a91-2f0ea9a12fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:37:48.186285 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b93965-f384-49cc-9a91-2f0ea9a12fe2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:37:48.187086 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b93965-f384-49cc-9a91-2f0ea9a12fe2 None admin] RBAC: Adding query filter params (role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac, scope.project.id=8dcda29e027a4e6ab74fcbfb53e24c5f, user.id=76dfbb65caa1403fbb797d0a8d4ce9cb) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:37:48.187419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b93965-f384-49cc-9a91-2f0ea9a12fe2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:37:48.188858 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b93965-f384-49cc-9a91-2f0ea9a12fe2 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:37:48.194451 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 594/1200] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:37:48 2022] GET /identity/v3/role_assignments?user.id=76dfbb65caa1403fbb797d0a8d4ce9cb&scope.project.id=8dcda29e027a4e6ab74fcbfb53e24c5f&role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 610 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:37:58.039305 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=32 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:37:58.191895 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=84 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:02.671691 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db34797-e6e7-4372-9cf7-e288dc1c0224 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:02.672289 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db34797-e6e7-4372-9cf7-e288dc1c0224 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:02.672289 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db34797-e6e7-4372-9cf7-e288dc1c0224 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:02.673580 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 607/1201] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:39:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:39:02.678859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6066070-443e-427d-9763-ca347e522ff7 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:02.679342 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6066070-443e-427d-9763-ca347e522ff7 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:02.679451 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6066070-443e-427d-9763-ca347e522ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:02.697633 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-e6066070-443e-427d-9763-ca347e522ff7 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:39:02.699178 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-e6066070-443e-427d-9763-ca347e522ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:39:02.767065 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 595/1202] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:39:02 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:39:02.791689 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db34797-e6e7-4372-9cf7-e288dc1c0224 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:02.894971 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db34797-e6e7-4372-9cf7-e288dc1c0224 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:02.932661 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f588ff86-b692-44b8-b09c-405be0c7eae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:02.933306 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588ff86-b692-44b8-b09c-405be0c7eae8 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:02.933588 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588ff86-b692-44b8-b09c-405be0c7eae8 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:02.933883 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f588ff86-b692-44b8-b09c-405be0c7eae8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:02.944675 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f588ff86-b692-44b8-b09c-405be0c7eae8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:02.946175 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f588ff86-b692-44b8-b09c-405be0c7eae8 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:02.961465 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-f588ff86-b692-44b8-b09c-405be0c7eae8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:39:02.961465 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:02.961465 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:02.961465 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:02.961465 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:02.961465 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:02.961465 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:02.961465 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:02.961465 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:02.961465 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:02.961465 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:02.961465 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:02.961465 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:39:02.961465 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:39:02.961465 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:39:02.965565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:39:02.965565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:02.965565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:02.965565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:02.965565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:02.965565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:02.965565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:02.965565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:02.965565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:02.965565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:02.965565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:02.965565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:02.965565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:02.965565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:02.965565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:02.965565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:02.965565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:02.966886 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:39:02.966886 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:39:02.966886 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:39:02.966886 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:39:02.966886 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 608/1203] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:39:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:39:02.969364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6066070-443e-427d-9763-ca347e522ff7 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:03.079958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6066070-443e-427d-9763-ca347e522ff7 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:03.110151 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:03.117190 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bb8b0c-e241-4716-b326-93b49bf1def8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:03.117845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb8b0c-e241-4716-b326-93b49bf1def8 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:03.118125 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb8b0c-e241-4716-b326-93b49bf1def8 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:03.118396 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bb8b0c-e241-4716-b326-93b49bf1def8 None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:03.119123 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb8b0c-e241-4716-b326-93b49bf1def8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:03.119429 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb8b0c-e241-4716-b326-93b49bf1def8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:03.120817 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bb8b0c-e241-4716-b326-93b49bf1def8 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:03.132149 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 596/1204] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:39:02 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:03.138213 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f588ff86-b692-44b8-b09c-405be0c7eae8 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:03.191565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:03.239829 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f588ff86-b692-44b8-b09c-405be0c7eae8 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:03.278213 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdbbb7e-ee12-478c-86d8-31c7f33eb31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:03.278835 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbbb7e-ee12-478c-86d8-31c7f33eb31c None admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:03.279144 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbbb7e-ee12-478c-86d8-31c7f33eb31c None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:03.279421 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdbbb7e-ee12-478c-86d8-31c7f33eb31c None admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:03.280109 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdbbb7e-ee12-478c-86d8-31c7f33eb31c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:03.281722 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdbbb7e-ee12-478c-86d8-31c7f33eb31c None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:03.444591 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 609/1205] 10.222.0.115 () {60 vars in 1137 bytes} [Tue Aug 2 10:39:03 2022] POST /identity/v3/users => generated 252 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:39:05.473899 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330509c-16a0-4c41-96f0-36535cf31e33 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:05.474959 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330509c-16a0-4c41-96f0-36535cf31e33 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:05.475082 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330509c-16a0-4c41-96f0-36535cf31e33 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:05.476575 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 597/1206] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:39:05 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:39:05.482460 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc5d7c-3224-41ec-9fcf-d096c292371d None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:05.482894 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc5d7c-3224-41ec-9fcf-d096c292371d None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:05.483280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfc5d7c-3224-41ec-9fcf-d096c292371d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:05.502691 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-dcfc5d7c-3224-41ec-9fcf-d096c292371d None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:39:05.503901 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-dcfc5d7c-3224-41ec-9fcf-d096c292371d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:39:05.580663 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 610/1207] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:39:05 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:39:05.605748 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330509c-16a0-4c41-96f0-36535cf31e33 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:05.718970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330509c-16a0-4c41-96f0-36535cf31e33 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:05.754050 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f8861-d943-4948-9a3b-a834b520318d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:05.754845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f8861-d943-4948-9a3b-a834b520318d None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:05.755148 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f8861-d943-4948-9a3b-a834b520318d None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:05.755420 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9f8861-d943-4948-9a3b-a834b520318d None admin] PATH_INFO: `/v3/roles/service` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:05.766951 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9f8861-d943-4948-9a3b-a834b520318d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:39:05.772692 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-fc9f8861-d943-4948-9a3b-a834b520318d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:39:05.772692 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:05.772692 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:05.772692 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:05.772692 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:05.772692 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:05.772692 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:05.772692 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:05.772692 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:05.772692 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:05.772692 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:05.772692 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:05.772692 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:39:05.772692 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:39:05.772692 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:39:05.772692 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:39:05.773962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:39:05.773962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:39:05.773962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:39:05.773962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:39:05.773962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:05.773962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:05.773962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:05.773962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:05.773962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:05.773962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:05.773962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:05.773962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:05.773962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:05.773962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:05.773962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:05.773962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:05.775343 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:05.775343 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:05.775343 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:05.775343 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:05.775343 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:39:05.775343 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:39:05.775343 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:39:05.775343 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:39:05.775343 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:39:05.775343 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:39:05.775343 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:39:05.775343 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:39:05.775343 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 598/1208] 10.222.0.115 () {56 vars in 1107 bytes} [Tue Aug 2 10:39:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:39:05.780121 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc5d7c-3224-41ec-9fcf-d096c292371d None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:05.821668 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:05.882504 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfc5d7c-3224-41ec-9fcf-d096c292371d None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:05.917050 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9773e4a0-f8f5-4c00-96e8-b3756b251657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:05.917672 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773e4a0-f8f5-4c00-96e8-b3756b251657 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:05.917957 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773e4a0-f8f5-4c00-96e8-b3756b251657 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:05.918235 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9773e4a0-f8f5-4c00-96e8-b3756b251657 None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:05.918963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9773e4a0-f8f5-4c00-96e8-b3756b251657 None admin] RBAC: Adding query filter params (name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:05.919299 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9773e4a0-f8f5-4c00-96e8-b3756b251657 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:05.920888 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9773e4a0-f8f5-4c00-96e8-b3756b251657 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:05.929854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 611/1209] 10.222.0.115 () {56 vars in 1108 bytes} [Tue Aug 2 10:39:05 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:05.935755 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f8861-d943-4948-9a3b-a834b520318d None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:05.987157 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:06.037173 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9f8861-d943-4948-9a3b-a834b520318d None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:06.072219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55e2bf-4aac-4a10-9c55-0a62083119a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:06.072845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55e2bf-4aac-4a10-9c55-0a62083119a6 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:06.073119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55e2bf-4aac-4a10-9c55-0a62083119a6 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:06.073386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55e2bf-4aac-4a10-9c55-0a62083119a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:06.084363 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55e2bf-4aac-4a10-9c55-0a62083119a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:06.085828 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55e2bf-4aac-4a10-9c55-0a62083119a6 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:06.101770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-ae55e2bf-4aac-4a10-9c55-0a62083119a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:39:06.101770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:06.101770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:06.101770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:06.101770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:06.101770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:06.101770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:06.101770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:06.101770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:06.101770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:06.101770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:06.101770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:06.101770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:39:06.101770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:39:06.101770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:39:06.102896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:39:06.102896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:06.102896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:06.102896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:06.102896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:06.102896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:06.102896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:06.102896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:06.102896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:06.102896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:06.102896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:06.102896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:06.102896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:06.102896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:06.102896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:06.102896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:06.102896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:06.104285 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:39:06.104285 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:39:06.104285 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:39:06.104285 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:39:06.104285 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 599/1210] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:39:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:39:06.108984 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9773e4a0-f8f5-4c00-96e8-b3756b251657 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:06.217415 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:06.220419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9773e4a0-f8f5-4c00-96e8-b3756b251657 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:06.256299 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2937d2a2-c5a4-490c-8cd7-f46c9b706017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:06.256832 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2937d2a2-c5a4-490c-8cd7-f46c9b706017 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:06.257114 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2937d2a2-c5a4-490c-8cd7-f46c9b706017 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:06.257391 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2937d2a2-c5a4-490c-8cd7-f46c9b706017 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:06.258088 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2937d2a2-c5a4-490c-8cd7-f46c9b706017 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:06.258389 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2937d2a2-c5a4-490c-8cd7-f46c9b706017 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:06.259825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2937d2a2-c5a4-490c-8cd7-f46c9b706017 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:06.272005 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 612/1211] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:39:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:06.277494 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55e2bf-4aac-4a10-9c55-0a62083119a6 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:06.380202 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55e2bf-4aac-4a10-9c55-0a62083119a6 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:06.381717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:06.422104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a73c9-db2f-4274-b647-0a886d936b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:06.423735 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a73c9-db2f-4274-b647-0a886d936b98 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:06.424434 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a73c9-db2f-4274-b647-0a886d936b98 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:06.425048 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1a73c9-db2f-4274-b647-0a886d936b98 None admin] PATH_INFO: `/v3/users/placement` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:06.438865 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a73c9-db2f-4274-b647-0a886d936b98 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:06.440326 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1a73c9-db2f-4274-b647-0a886d936b98 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:06.452345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-8f1a73c9-db2f-4274-b647-0a886d936b98 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:39:06.452345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:06.452345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:06.452345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:06.452345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:06.452345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:06.452345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:06.452345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:06.452345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:06.452345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:06.452345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:06.452345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:06.452345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:39:06.452345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:39:06.452345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:39:06.452345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:39:06.453563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:06.453563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:06.453563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:39:06.453563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:39:06.453563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:39:06.453563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:39:06.453563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:06.453563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:06.453563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:06.453563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:06.453563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:06.453563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:06.453563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:06.453563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:06.453563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:06.453563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:06.453563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:06.453563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:06.454714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:06.454714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:06.454714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:39:06.454714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:39:06.454714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:39:06.454714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:39:06.454714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:39:06.454714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:39:06.454714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:39:06.454714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:39:06.454714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 600/1212] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:39:06 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:39:06.460019 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2937d2a2-c5a4-490c-8cd7-f46c9b706017 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:06.563149 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2937d2a2-c5a4-490c-8cd7-f46c9b706017 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:06.599160 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c13c94-c048-4a82-90ad-941db6056622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:06.599772 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c13c94-c048-4a82-90ad-941db6056622 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:06.600052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c13c94-c048-4a82-90ad-941db6056622 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:06.600339 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c13c94-c048-4a82-90ad-941db6056622 None admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:06.601123 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c13c94-c048-4a82-90ad-941db6056622 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:06.601429 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c13c94-c048-4a82-90ad-941db6056622 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:06.602951 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c13c94-c048-4a82-90ad-941db6056622 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:06.621284 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 613/1213] 10.222.0.115 () {56 vars in 1148 bytes} [Tue Aug 2 10:39:06 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:06.627289 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a73c9-db2f-4274-b647-0a886d936b98 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:06.728521 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1a73c9-db2f-4274-b647-0a886d936b98 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:06.762996 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96d006-e656-491c-9f6d-6d7812f62191 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:06.763632 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96d006-e656-491c-9f6d-6d7812f62191 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:06.763911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96d006-e656-491c-9f6d-6d7812f62191 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:06.764180 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96d006-e656-491c-9f6d-6d7812f62191 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:06.796659 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:06.836859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a96d006-e656-491c-9f6d-6d7812f62191 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:06.838512 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a96d006-e656-491c-9f6d-6d7812f62191 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:06.953694 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 601/1214] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:39:06 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:39:06.960490 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c13c94-c048-4a82-90ad-941db6056622 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:06.982298 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:07.064886 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c13c94-c048-4a82-90ad-941db6056622 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:07.101899 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd898ed-b5a6-46d4-894d-588163b57d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:07.102458 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd898ed-b5a6-46d4-894d-588163b57d13 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:07.102739 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd898ed-b5a6-46d4-894d-588163b57d13 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:07.103011 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd898ed-b5a6-46d4-894d-588163b57d13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:07.113469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd898ed-b5a6-46d4-894d-588163b57d13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:07.115451 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd898ed-b5a6-46d4-894d-588163b57d13 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:07.130934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-5bd898ed-b5a6-46d4-894d-588163b57d13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:39:07.130934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:07.130934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:07.130934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:07.130934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:07.130934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:07.130934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:07.130934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:07.130934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:07.130934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:07.130934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:07.130934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:07.130934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:39:07.130934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:39:07.130934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:39:07.132077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:39:07.132077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:07.132077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:07.132077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:07.132077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:07.132077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:07.132077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:07.132077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:07.132077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:07.132077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:07.132077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:07.132077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:07.132077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:07.132077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:07.132077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:07.132077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:07.132077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:07.133238 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:39:07.133238 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:39:07.133238 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:39:07.133238 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:39:07.133238 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 614/1215] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:39:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:39:07.137898 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96d006-e656-491c-9f6d-6d7812f62191 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:07.200404 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:07.237537 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96d006-e656-491c-9f6d-6d7812f62191 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:07.272256 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f44209c-f81e-4b71-9e86-7005aaff7526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:07.272864 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44209c-f81e-4b71-9e86-7005aaff7526 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:07.273145 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44209c-f81e-4b71-9e86-7005aaff7526 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:07.273417 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f44209c-f81e-4b71-9e86-7005aaff7526 None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:07.274105 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f44209c-f81e-4b71-9e86-7005aaff7526 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:07.274407 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f44209c-f81e-4b71-9e86-7005aaff7526 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:07.275801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f44209c-f81e-4b71-9e86-7005aaff7526 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:07.288339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 602/1216] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:39:07 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:07.294276 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd898ed-b5a6-46d4-894d-588163b57d13 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:07.371332 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:07.404247 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd898ed-b5a6-46d4-894d-588163b57d13 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:07.442244 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87154bb4-ee74-4a71-9002-aa004dd7f561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:07.443018 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87154bb4-ee74-4a71-9002-aa004dd7f561 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:07.443240 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87154bb4-ee74-4a71-9002-aa004dd7f561 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:07.443513 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87154bb4-ee74-4a71-9002-aa004dd7f561 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:07.450912 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87154bb4-ee74-4a71-9002-aa004dd7f561 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:07.452443 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87154bb4-ee74-4a71-9002-aa004dd7f561 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:07.464574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-87154bb4-ee74-4a71-9002-aa004dd7f561 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:39:07.464574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:07.464574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:07.464574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:07.464574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:07.464574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:07.464574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:07.464574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:07.464574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:07.464574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:07.464574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:07.464574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:07.464574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:39:07.464574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:39:07.464574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:39:07.466016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:39:07.466016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:07.466016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:07.466016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:07.466016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:07.466016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:07.466016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:07.466016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:07.466016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:07.466016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:07.466016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:07.466016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:07.466016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:07.466016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:07.466016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:07.466016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:07.466016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:07.467715 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:39:07.467715 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:39:07.467715 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:39:07.467715 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:39:07.467715 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:39:07.467715 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:39:07.467715 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:39:07.467715 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:39:07.467715 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 615/1217] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:39:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:39:07.472288 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f44209c-f81e-4b71-9e86-7005aaff7526 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:07.580405 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f44209c-f81e-4b71-9e86-7005aaff7526 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:07.603589 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:07.616048 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8306234-6009-4a6f-a0e1-a90a1ae9a010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:07.616665 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8306234-6009-4a6f-a0e1-a90a1ae9a010 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:07.616973 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8306234-6009-4a6f-a0e1-a90a1ae9a010 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:07.617246 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8306234-6009-4a6f-a0e1-a90a1ae9a010 None admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:07.617997 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8306234-6009-4a6f-a0e1-a90a1ae9a010 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:07.618300 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8306234-6009-4a6f-a0e1-a90a1ae9a010 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:07.619801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8306234-6009-4a6f-a0e1-a90a1ae9a010 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:07.633940 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 603/1218] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:39:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:07.639345 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87154bb4-ee74-4a71-9002-aa004dd7f561 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:07.742643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87154bb4-ee74-4a71-9002-aa004dd7f561 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:07.771543 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:07.777280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39becacc-7136-4d53-9278-12ff9d7b5b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:07.777905 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39becacc-7136-4d53-9278-12ff9d7b5b93 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:07.778189 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39becacc-7136-4d53-9278-12ff9d7b5b93 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:07.778465 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39becacc-7136-4d53-9278-12ff9d7b5b93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:07.779276 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39becacc-7136-4d53-9278-12ff9d7b5b93 None admin] RBAC: Adding query filter params (role.id=0f1c61dab22c4d97ae784468058036c7, scope.project.id=c1eae881e5714bf4bd223809e1a2d920, user.id=cbc4b962e2a742b08beef7187cc527e5) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:07.779608 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39becacc-7136-4d53-9278-12ff9d7b5b93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:07.781065 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39becacc-7136-4d53-9278-12ff9d7b5b93 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:07.787030 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 616/1219] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:39:07 2022] GET /identity/v3/role_assignments?user.id=cbc4b962e2a742b08beef7187cc527e5&scope.project.id=c1eae881e5714bf4bd223809e1a2d920&role.id=0f1c61dab22c4d97ae784468058036c7 => generated 260 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:09.811160 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fef201-7ca1-4870-8eb4-784fb1723e36 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:09.811759 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fef201-7ca1-4870-8eb4-784fb1723e36 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:09.811759 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fef201-7ca1-4870-8eb4-784fb1723e36 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:09.813104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 604/1220] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:39:09 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:39:09.818508 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448bfaf8-27b9-45a2-8474-29aa61d1eebb None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:09.818909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448bfaf8-27b9-45a2-8474-29aa61d1eebb None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:09.819074 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448bfaf8-27b9-45a2-8474-29aa61d1eebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:09.837749 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-448bfaf8-27b9-45a2-8474-29aa61d1eebb None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:39:09.838943 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-448bfaf8-27b9-45a2-8474-29aa61d1eebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:39:09.912473 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 617/1221] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:39:09 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:39:09.935610 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fef201-7ca1-4870-8eb4-784fb1723e36 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:10.044059 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fef201-7ca1-4870-8eb4-784fb1723e36 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:10.078374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cee1f5-b268-48fa-b03e-bf21dde3f0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:10.078930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cee1f5-b268-48fa-b03e-bf21dde3f0fe None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:10.079237 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cee1f5-b268-48fa-b03e-bf21dde3f0fe None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:10.079512 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cee1f5-b268-48fa-b03e-bf21dde3f0fe None admin] PATH_INFO: `/v3/roles/service` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:10.091000 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cee1f5-b268-48fa-b03e-bf21dde3f0fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:39:10.095920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-b9cee1f5-b268-48fa-b03e-bf21dde3f0fe None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:39:10.095920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:10.095920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:10.095920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:10.095920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:10.095920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:10.095920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:10.095920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:10.095920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:10.095920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:10.095920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:10.095920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:10.095920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:39:10.095920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:39:10.095920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:39:10.095920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:39:10.096912 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:39:10.096912 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:39:10.096912 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:39:10.096912 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:39:10.096912 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:10.096912 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:10.096912 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:10.096912 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:10.096912 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:10.096912 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:10.096912 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:10.096912 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:10.096912 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:10.096912 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:10.096912 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:10.096912 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:10.098071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:10.098071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:10.098071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:10.098071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:10.098071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:39:10.098071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:39:10.098071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:39:10.098071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:39:10.098071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:39:10.098071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:39:10.098071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:39:10.098071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:39:10.098071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 605/1222] 10.222.0.115 () {56 vars in 1107 bytes} [Tue Aug 2 10:39:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:39:10.104177 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448bfaf8-27b9-45a2-8474-29aa61d1eebb None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:10.205685 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448bfaf8-27b9-45a2-8474-29aa61d1eebb None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:10.230187 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:10.240842 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbded20-a072-41f5-8269-1cb6e2a158e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:10.241460 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbded20-a072-41f5-8269-1cb6e2a158e8 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:10.241740 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbded20-a072-41f5-8269-1cb6e2a158e8 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:10.242010 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbded20-a072-41f5-8269-1cb6e2a158e8 None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:10.242737 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbded20-a072-41f5-8269-1cb6e2a158e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:10.243019 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbded20-a072-41f5-8269-1cb6e2a158e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:10.244410 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbded20-a072-41f5-8269-1cb6e2a158e8 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:10.252187 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 618/1223] 10.222.0.115 () {56 vars in 1108 bytes} [Tue Aug 2 10:39:10 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:10.258155 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cee1f5-b268-48fa-b03e-bf21dde3f0fe None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:10.303240 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:10.358648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cee1f5-b268-48fa-b03e-bf21dde3f0fe None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:10.393050 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d01432a-4227-4850-ac38-de676226a06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:10.393675 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01432a-4227-4850-ac38-de676226a06d None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:10.393953 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01432a-4227-4850-ac38-de676226a06d None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:10.394224 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d01432a-4227-4850-ac38-de676226a06d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:10.404380 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d01432a-4227-4850-ac38-de676226a06d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:10.405824 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d01432a-4227-4850-ac38-de676226a06d None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:10.420843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-9d01432a-4227-4850-ac38-de676226a06d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:39:10.420843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:10.420843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:10.420843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:10.420843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:10.420843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:10.420843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:10.420843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:10.420843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:10.420843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:10.420843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:10.420843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:10.420843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:39:10.420843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:39:10.420843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:39:10.421966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:39:10.421966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:10.421966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:10.421966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:10.421966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:10.421966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:10.421966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:10.421966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:10.421966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:10.421966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:10.421966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:10.421966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:10.421966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:10.421966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:10.421966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:10.421966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:10.421966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:10.422907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:39:10.422907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:39:10.422907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:39:10.422907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:39:10.422907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 606/1224] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:39:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:39:10.428694 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbded20-a072-41f5-8269-1cb6e2a158e8 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:10.532933 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbded20-a072-41f5-8269-1cb6e2a158e8 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:10.566170 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703fafe-f249-4698-a759-a0647a61c7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:10.566795 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703fafe-f249-4698-a759-a0647a61c7b7 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:10.567242 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703fafe-f249-4698-a759-a0647a61c7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:10.567521 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703fafe-f249-4698-a759-a0647a61c7b7 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:10.568179 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e703fafe-f249-4698-a759-a0647a61c7b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:10.568472 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e703fafe-f249-4698-a759-a0647a61c7b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:10.569816 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e703fafe-f249-4698-a759-a0647a61c7b7 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:10.582195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 619/1225] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:39:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:39:10.588182 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d01432a-4227-4850-ac38-de676226a06d None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:10.682157 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:10.688551 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d01432a-4227-4850-ac38-de676226a06d None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:10.721557 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d6c15-7b80-412d-9402-d7048d4f9145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:10.722185 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d6c15-7b80-412d-9402-d7048d4f9145 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:10.722458 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d6c15-7b80-412d-9402-d7048d4f9145 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:10.722724 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316d6c15-7b80-412d-9402-d7048d4f9145 None admin] PATH_INFO: `/v3/users/placement` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:10.730694 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d6c15-7b80-412d-9402-d7048d4f9145 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:10.732156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316d6c15-7b80-412d-9402-d7048d4f9145 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:10.743286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-316d6c15-7b80-412d-9402-d7048d4f9145 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:39:10.743286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:10.743286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:10.743286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:10.743286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:10.743286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:10.743286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:10.743286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:10.743286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:10.743286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:10.743286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:10.743286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:10.743286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:39:10.743286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:39:10.743286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:39:10.743286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:39:10.744948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:10.744948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:10.744948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:39:10.744948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:39:10.744948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:39:10.744948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:39:10.744948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:10.744948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:10.744948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:10.744948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:10.744948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:10.744948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:10.744948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:10.744948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:10.744948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:10.744948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:10.744948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:10.744948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:10.746016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:10.746016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:10.746016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:39:10.746016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:39:10.746016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:39:10.746016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:39:10.746016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:39:10.746016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:39:10.746016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:39:10.746016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:39:10.746016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 607/1226] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:39:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:39:10.750326 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703fafe-f249-4698-a759-a0647a61c7b7 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:10.797990 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:10.851029 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703fafe-f249-4698-a759-a0647a61c7b7 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:10.884468 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5bc0c4-17c0-454e-b482-67bff4769f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:10.885076 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bc0c4-17c0-454e-b482-67bff4769f3f None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:10.885361 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bc0c4-17c0-454e-b482-67bff4769f3f None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:10.885634 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5bc0c4-17c0-454e-b482-67bff4769f3f None admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:10.886419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5bc0c4-17c0-454e-b482-67bff4769f3f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:10.886722 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5bc0c4-17c0-454e-b482-67bff4769f3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:10.888208 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5bc0c4-17c0-454e-b482-67bff4769f3f None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:10.907283 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 620/1227] 10.222.0.115 () {56 vars in 1148 bytes} [Tue Aug 2 10:39:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:10.913839 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d6c15-7b80-412d-9402-d7048d4f9145 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:11.015248 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316d6c15-7b80-412d-9402-d7048d4f9145 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:11.048551 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf0fef-3b44-4296-be02-fc0185f0d201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:11.049167 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf0fef-3b44-4296-be02-fc0185f0d201 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:11.049445 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf0fef-3b44-4296-be02-fc0185f0d201 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:11.049712 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdf0fef-3b44-4296-be02-fc0185f0d201 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:11.059988 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdf0fef-3b44-4296-be02-fc0185f0d201 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:11.061415 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fdf0fef-3b44-4296-be02-fc0185f0d201 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:11.075455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-5fdf0fef-3b44-4296-be02-fc0185f0d201 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:39:11.075455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:11.075455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:11.075455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:11.075455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:11.075455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:11.075455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:11.075455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:11.075455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:11.075455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:11.075455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:11.075455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:11.075455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:39:11.075455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:39:11.075455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:39:11.076582 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:39:11.076582 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:11.076582 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:11.076582 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:11.076582 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:11.076582 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:11.076582 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:11.076582 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:11.076582 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:11.076582 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:11.076582 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:11.076582 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:11.076582 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:11.076582 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:11.076582 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:11.076582 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:11.076582 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:11.077823 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:39:11.077823 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:39:11.077823 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:39:11.077823 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:39:11.077823 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 608/1228] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:39:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:39:11.083024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5bc0c4-17c0-454e-b482-67bff4769f3f None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:11.184559 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5bc0c4-17c0-454e-b482-67bff4769f3f None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:11.186989 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:11.218494 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861219a-0dc5-4f82-bc8e-3737a8ca6631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:11.219139 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861219a-0dc5-4f82-bc8e-3737a8ca6631 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:11.219419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861219a-0dc5-4f82-bc8e-3737a8ca6631 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:11.219691 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2861219a-0dc5-4f82-bc8e-3737a8ca6631 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:11.220372 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861219a-0dc5-4f82-bc8e-3737a8ca6631 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:11.220669 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861219a-0dc5-4f82-bc8e-3737a8ca6631 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:11.222023 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2861219a-0dc5-4f82-bc8e-3737a8ca6631 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:11.233902 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 621/1229] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:39:11 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:11.240716 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf0fef-3b44-4296-be02-fc0185f0d201 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:11.250174 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:11.343491 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdf0fef-3b44-4296-be02-fc0185f0d201 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:11.378019 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c9247-d7c8-4398-a6a5-1a1265f7f5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:11.378690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c9247-d7c8-4398-a6a5-1a1265f7f5db None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:11.378995 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c9247-d7c8-4398-a6a5-1a1265f7f5db None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:11.379355 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341c9247-d7c8-4398-a6a5-1a1265f7f5db None admin] PATH_INFO: `/v3/projects/service` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:11.386176 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c9247-d7c8-4398-a6a5-1a1265f7f5db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:11.387643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341c9247-d7c8-4398-a6a5-1a1265f7f5db None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:11.399220 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-341c9247-d7c8-4398-a6a5-1a1265f7f5db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:39:11.399220 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:11.399220 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:11.399220 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:11.399220 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:11.399220 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:11.399220 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:11.399220 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:11.399220 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:11.399220 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:11.399220 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:11.399220 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:11.399220 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:39:11.399220 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:39:11.399220 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:39:11.400206 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:39:11.400206 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:11.400206 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:11.400206 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:11.400206 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:11.400206 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:11.400206 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:11.400206 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:11.400206 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:11.400206 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:11.400206 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:11.400206 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:11.400206 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:11.400206 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:11.400206 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:11.400206 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:11.400206 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:11.401352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:39:11.401352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:39:11.401352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:39:11.401352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:39:11.401352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:39:11.401352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:39:11.401352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:39:11.401352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:39:11.401352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 609/1230] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:39:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:39:11.406705 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861219a-0dc5-4f82-bc8e-3737a8ca6631 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:11.532139 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2861219a-0dc5-4f82-bc8e-3737a8ca6631 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:11.566164 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43187a2-2869-48f0-9bfb-024eeb388219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:11.566796 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43187a2-2869-48f0-9bfb-024eeb388219 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:11.567211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43187a2-2869-48f0-9bfb-024eeb388219 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:11.567714 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43187a2-2869-48f0-9bfb-024eeb388219 None admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:11.568488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43187a2-2869-48f0-9bfb-024eeb388219 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:11.568796 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43187a2-2869-48f0-9bfb-024eeb388219 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:11.570210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43187a2-2869-48f0-9bfb-024eeb388219 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:11.583004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 622/1231] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:39:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:11.589290 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c9247-d7c8-4398-a6a5-1a1265f7f5db None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:11.655934 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:11.691385 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341c9247-d7c8-4398-a6a5-1a1265f7f5db None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:11.726348 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcb0006-2556-40bb-a693-77babdd193d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:11.727015 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb0006-2556-40bb-a693-77babdd193d4 None admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:11.727323 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb0006-2556-40bb-a693-77babdd193d4 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:11.727597 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcb0006-2556-40bb-a693-77babdd193d4 None admin] PATH_INFO: `/v3/projects/c1eae881e5714bf4bd223809e1a2d920/users/cbc4b962e2a742b08beef7187cc527e5/roles/0f1c61dab22c4d97ae784468058036c7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:11.743396 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb0006-2556-40bb-a693-77babdd193d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=c1eae881e5714bf4bd223809e1a2d920, user_id=cbc4b962e2a742b08beef7187cc527e5, role_id=0f1c61dab22c4d97ae784468058036c7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:11.744870 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcb0006-2556-40bb-a693-77babdd193d4 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:11.896444 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 610/1232] 10.222.0.115 () {58 vars in 1444 bytes} [Tue Aug 2 10:39:11 2022] PUT /identity/v3/projects/c1eae881e5714bf4bd223809e1a2d920/users/cbc4b962e2a742b08beef7187cc527e5/roles/0f1c61dab22c4d97ae784468058036c7 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:39:13.967999 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa25ec-0ea8-49d5-90ca-1be178e93775 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:13.968585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa25ec-0ea8-49d5-90ca-1be178e93775 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:13.968585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40aa25ec-0ea8-49d5-90ca-1be178e93775 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:13.969954 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 623/1233] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:39:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:39:13.976149 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd196c24-a64c-42ee-80c0-503aca6292f9 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:13.976663 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd196c24-a64c-42ee-80c0-503aca6292f9 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:13.976758 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd196c24-a64c-42ee-80c0-503aca6292f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:13.995604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-dd196c24-a64c-42ee-80c0-503aca6292f9 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:39:13.996849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-dd196c24-a64c-42ee-80c0-503aca6292f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:39:14.068483 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 611/1234] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:39:13 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:39:14.093134 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aa25ec-0ea8-49d5-90ca-1be178e93775 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:14.119552 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:14.198216 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40aa25ec-0ea8-49d5-90ca-1be178e93775 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:14.234371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052ebac-5355-4b26-92b3-6361ff6fd337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:14.234986 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052ebac-5355-4b26-92b3-6361ff6fd337 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:14.235296 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052ebac-5355-4b26-92b3-6361ff6fd337 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:14.240364 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052ebac-5355-4b26-92b3-6361ff6fd337 None admin] PATH_INFO: `/v3/roles/service` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:14.245994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4052ebac-5355-4b26-92b3-6361ff6fd337 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:39:14.250850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-4052ebac-5355-4b26-92b3-6361ff6fd337 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:39:14.250850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:14.250850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:14.250850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:14.250850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:14.250850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:14.250850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:14.250850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:14.250850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:14.250850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:14.250850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:14.250850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:14.250850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:39:14.250850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:39:14.250850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:39:14.250850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:39:14.251918 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:39:14.251918 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:39:14.251918 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:39:14.251918 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:39:14.251918 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:14.251918 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:14.251918 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:14.251918 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:14.251918 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:14.251918 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:14.251918 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:14.251918 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:14.251918 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:14.251918 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:14.251918 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:14.251918 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:14.252794 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:14.252794 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:14.252794 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:14.252794 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:14.252794 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:39:14.252794 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:39:14.252794 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:39:14.252794 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:39:14.252794 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:39:14.252794 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:39:14.252794 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 02 10:39:14.252794 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:39:14.252794 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 624/1235] 10.222.0.115 () {56 vars in 1107 bytes} [Tue Aug 2 10:39:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:39:14.258792 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd196c24-a64c-42ee-80c0-503aca6292f9 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:14.306798 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:14.362374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd196c24-a64c-42ee-80c0-503aca6292f9 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:14.397907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9054ee3d-e900-4e48-9231-5c6e4e7f374a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:14.398528 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054ee3d-e900-4e48-9231-5c6e4e7f374a None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:14.398800 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054ee3d-e900-4e48-9231-5c6e4e7f374a None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:14.399088 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9054ee3d-e900-4e48-9231-5c6e4e7f374a None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:14.399810 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9054ee3d-e900-4e48-9231-5c6e4e7f374a None admin] RBAC: Adding query filter params (name=service) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:14.400108 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9054ee3d-e900-4e48-9231-5c6e4e7f374a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:14.401493 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9054ee3d-e900-4e48-9231-5c6e4e7f374a None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:14.409995 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 612/1236] 10.222.0.115 () {56 vars in 1108 bytes} [Tue Aug 2 10:39:14 2022] GET /identity/v3/roles?name=service => generated 328 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:14.416024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052ebac-5355-4b26-92b3-6361ff6fd337 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:14.517071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:14.532737 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052ebac-5355-4b26-92b3-6361ff6fd337 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:14.566059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e6e6c6-084a-483f-aef5-f5bd7e483e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:14.567419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6e6c6-084a-483f-aef5-f5bd7e483e0c None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:14.567733 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6e6c6-084a-483f-aef5-f5bd7e483e0c None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:14.568009 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e6e6c6-084a-483f-aef5-f5bd7e483e0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:14.578840 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e6e6c6-084a-483f-aef5-f5bd7e483e0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:14.580345 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e6e6c6-084a-483f-aef5-f5bd7e483e0c None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:14.594731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-c7e6e6c6-084a-483f-aef5-f5bd7e483e0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:39:14.594731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:14.594731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:14.594731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:14.594731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:14.594731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:14.594731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:14.594731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:14.594731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:14.594731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:14.594731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:14.594731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:14.594731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:39:14.594731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:39:14.594731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:39:14.598595 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:39:14.598595 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:14.598595 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:14.598595 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:14.598595 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:14.598595 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:14.598595 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:14.598595 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:14.598595 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:14.598595 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:14.598595 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:14.598595 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:14.598595 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:14.598595 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:14.598595 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:14.598595 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:14.598595 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:14.599779 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:39:14.599779 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:39:14.599779 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:39:14.599779 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:39:14.599779 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 625/1237] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:39:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:39:14.601929 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9054ee3d-e900-4e48-9231-5c6e4e7f374a None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:14.705983 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9054ee3d-e900-4e48-9231-5c6e4e7f374a None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:14.715668 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:14.741746 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e6eee1-1d84-4583-9534-cfa656d7b0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:14.742308 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6eee1-1d84-4583-9534-cfa656d7b0e3 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:14.742582 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6eee1-1d84-4583-9534-cfa656d7b0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:14.742848 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e6eee1-1d84-4583-9534-cfa656d7b0e3 None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:14.743562 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e6eee1-1d84-4583-9534-cfa656d7b0e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:14.743861 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e6eee1-1d84-4583-9534-cfa656d7b0e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:14.745211 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e6eee1-1d84-4583-9534-cfa656d7b0e3 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:14.758517 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 613/1238] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:39:14 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:14.764316 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e6e6c6-084a-483f-aef5-f5bd7e483e0c None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:14.870186 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e6e6c6-084a-483f-aef5-f5bd7e483e0c None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:14.905536 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97f2f47-9eff-46b3-86e1-42b9d5ece202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:14.906278 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f2f47-9eff-46b3-86e1-42b9d5ece202 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:14.906593 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f2f47-9eff-46b3-86e1-42b9d5ece202 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:14.906867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97f2f47-9eff-46b3-86e1-42b9d5ece202 None admin] PATH_INFO: `/v3/users/placement` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:14.910026 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:14.915975 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97f2f47-9eff-46b3-86e1-42b9d5ece202 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:14.917599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97f2f47-9eff-46b3-86e1-42b9d5ece202 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:14.936132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-d97f2f47-9eff-46b3-86e1-42b9d5ece202 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:39:14.936132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:14.936132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:14.936132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:14.936132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:14.936132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:14.936132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:14.936132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:14.936132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:14.936132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:14.936132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:14.936132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:14.936132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:39:14.936132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:39:14.936132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:39:14.936132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:39:14.940858 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:14.940858 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:14.940858 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:39:14.940858 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:39:14.940858 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:39:14.940858 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:39:14.940858 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:14.940858 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:14.940858 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:14.940858 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:14.940858 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:14.940858 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:14.940858 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:14.940858 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:14.940858 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:14.940858 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:14.940858 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:14.940858 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:14.943198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:14.943198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:14.943198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:39:14.943198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:39:14.943198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:39:14.943198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:39:14.943198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:39:14.943198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:39:14.943198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:39:14.943198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:39:14.943198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 626/1239] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:39:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:39:14.947739 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e6eee1-1d84-4583-9534-cfa656d7b0e3 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:15.053675 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e6eee1-1d84-4583-9534-cfa656d7b0e3 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:15.089428 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27206c38-87b5-4a1f-a407-634d8a0f1597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:15.090165 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27206c38-87b5-4a1f-a407-634d8a0f1597 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:15.090517 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27206c38-87b5-4a1f-a407-634d8a0f1597 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:15.090800 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27206c38-87b5-4a1f-a407-634d8a0f1597 None admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:15.091639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27206c38-87b5-4a1f-a407-634d8a0f1597 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:15.091978 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27206c38-87b5-4a1f-a407-634d8a0f1597 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:15.093419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27206c38-87b5-4a1f-a407-634d8a0f1597 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:15.111909 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 614/1240] 10.222.0.115 () {56 vars in 1148 bytes} [Tue Aug 2 10:39:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:15.118078 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97f2f47-9eff-46b3-86e1-42b9d5ece202 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:15.227390 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97f2f47-9eff-46b3-86e1-42b9d5ece202 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:15.262850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e56ad-7c29-4d5b-8f06-11d50900cd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:15.263560 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e56ad-7c29-4d5b-8f06-11d50900cd80 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:15.263841 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e56ad-7c29-4d5b-8f06-11d50900cd80 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:15.264112 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e56ad-7c29-4d5b-8f06-11d50900cd80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:15.328329 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:15.335341 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e56ad-7c29-4d5b-8f06-11d50900cd80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:15.336794 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06e56ad-7c29-4d5b-8f06-11d50900cd80 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:15.436325 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 627/1241] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:39:15 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 10:39:15.442999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27206c38-87b5-4a1f-a407-634d8a0f1597 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:15.471493 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:15.545703 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27206c38-87b5-4a1f-a407-634d8a0f1597 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:15.579482 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cfabd3-7032-4dcf-a632-11efd1f7fb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:15.580108 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cfabd3-7032-4dcf-a632-11efd1f7fb14 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:15.580386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cfabd3-7032-4dcf-a632-11efd1f7fb14 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:15.580658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cfabd3-7032-4dcf-a632-11efd1f7fb14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:15.591078 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cfabd3-7032-4dcf-a632-11efd1f7fb14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:15.592517 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cfabd3-7032-4dcf-a632-11efd1f7fb14 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:15.607369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-40cfabd3-7032-4dcf-a632-11efd1f7fb14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:39:15.607369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:15.607369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:15.607369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:15.607369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:15.607369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:15.607369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:15.607369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:15.607369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:15.607369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:15.607369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:15.607369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:15.607369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:39:15.607369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:39:15.607369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:39:15.608504 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:39:15.608504 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:15.608504 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:15.608504 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:15.608504 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:15.608504 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:15.608504 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:15.608504 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:15.608504 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:15.608504 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:15.608504 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:15.608504 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:15.608504 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:15.608504 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:15.608504 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:15.608504 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:15.608504 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:15.609643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:39:15.609643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:39:15.609643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:39:15.609643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:39:15.609643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 615/1242] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:39:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:39:15.614427 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e56ad-7c29-4d5b-8f06-11d50900cd80 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:15.710754 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:15.716526 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e56ad-7c29-4d5b-8f06-11d50900cd80 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:15.752266 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150e359-6ae0-4da2-b4e4-8f03f514479c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:15.752875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150e359-6ae0-4da2-b4e4-8f03f514479c None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:15.753155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150e359-6ae0-4da2-b4e4-8f03f514479c None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:15.753473 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150e359-6ae0-4da2-b4e4-8f03f514479c None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:15.754206 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150e359-6ae0-4da2-b4e4-8f03f514479c None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:15.754516 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150e359-6ae0-4da2-b4e4-8f03f514479c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:15.755912 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150e359-6ae0-4da2-b4e4-8f03f514479c None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:15.767123 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 628/1243] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:39:15 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:15.772931 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cfabd3-7032-4dcf-a632-11efd1f7fb14 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:15.855368 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:15.881695 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cfabd3-7032-4dcf-a632-11efd1f7fb14 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:15.914793 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae42ce4-2c4a-4a0c-8a8a-b5ef11eb244d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:15.915452 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae42ce4-2c4a-4a0c-8a8a-b5ef11eb244d None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:15.915732 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae42ce4-2c4a-4a0c-8a8a-b5ef11eb244d None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:15.915996 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae42ce4-2c4a-4a0c-8a8a-b5ef11eb244d None admin] PATH_INFO: `/v3/projects/service` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:15.922497 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae42ce4-2c4a-4a0c-8a8a-b5ef11eb244d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:15.924088 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae42ce4-2c4a-4a0c-8a8a-b5ef11eb244d None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:15.935937 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-eae42ce4-2c4a-4a0c-8a8a-b5ef11eb244d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:39:15.935937 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:15.935937 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:15.935937 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:15.935937 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:15.935937 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:15.935937 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:15.935937 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:15.935937 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:15.935937 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:15.935937 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:15.935937 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:15.935937 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:39:15.935937 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:39:15.935937 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:39:15.937144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:39:15.937144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:15.937144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:15.937144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:15.937144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:15.937144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:15.937144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:15.937144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:15.937144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:15.937144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:15.937144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:15.937144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:15.937144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:15.937144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:15.937144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:15.937144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:15.937144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:15.938073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:39:15.938073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:39:15.938073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:39:15.938073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:39:15.938073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:39:15.938073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:39:15.938073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:39:15.938073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:39:15.939394 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 616/1244] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:39:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:39:15.943650 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150e359-6ae0-4da2-b4e4-8f03f514479c None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:16.047609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150e359-6ae0-4da2-b4e4-8f03f514479c None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:16.081619 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76c38e0-3c12-40aa-bf29-07592d7f38ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:16.082831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c38e0-3c12-40aa-bf29-07592d7f38ce None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:16.083131 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c38e0-3c12-40aa-bf29-07592d7f38ce None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:16.083471 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76c38e0-3c12-40aa-bf29-07592d7f38ce None admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:16.084229 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c38e0-3c12-40aa-bf29-07592d7f38ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:16.084540 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c38e0-3c12-40aa-bf29-07592d7f38ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:16.085970 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b76c38e0-3c12-40aa-bf29-07592d7f38ce None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:16.099749 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 629/1245] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:39:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:16.105765 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae42ce4-2c4a-4a0c-8a8a-b5ef11eb244d None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:16.210817 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae42ce4-2c4a-4a0c-8a8a-b5ef11eb244d None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:16.244661 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e0853-a1e1-42d1-a1be-c3809b929ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:16.245272 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e0853-a1e1-42d1-a1be-c3809b929ac1 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:16.245546 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e0853-a1e1-42d1-a1be-c3809b929ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:16.245820 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e0853-a1e1-42d1-a1be-c3809b929ac1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:16.246651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e0853-a1e1-42d1-a1be-c3809b929ac1 None admin] RBAC: Adding query filter params (role.id=0f1c61dab22c4d97ae784468058036c7, scope.project.id=c1eae881e5714bf4bd223809e1a2d920, user.id=cbc4b962e2a742b08beef7187cc527e5) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:16.246962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e0853-a1e1-42d1-a1be-c3809b929ac1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:16.248520 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e0853-a1e1-42d1-a1be-c3809b929ac1 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:16.254612 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 617/1246] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:39:16 2022] GET /identity/v3/role_assignments?user.id=cbc4b962e2a742b08beef7187cc527e5&scope.project.id=c1eae881e5714bf4bd223809e1a2d920&role.id=0f1c61dab22c4d97ae784468058036c7 => generated 610 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:18.348676 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e09c8-4a90-446a-9b52-b634c1bf8758 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:18.349233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e09c8-4a90-446a-9b52-b634c1bf8758 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:18.349233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e09c8-4a90-446a-9b52-b634c1bf8758 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:18.350609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 630/1247] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:39:18 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:39:18.355716 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4011aac-d220-4caa-8fd8-0044a3e396d9 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:18.356144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4011aac-d220-4caa-8fd8-0044a3e396d9 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:18.356366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4011aac-d220-4caa-8fd8-0044a3e396d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:18.362622 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:18.375407 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-d4011aac-d220-4caa-8fd8-0044a3e396d9 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:39:18.376579 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-d4011aac-d220-4caa-8fd8-0044a3e396d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:39:18.456623 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 618/1248] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:39:18 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:39:18.480150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e09c8-4a90-446a-9b52-b634c1bf8758 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:18.498876 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:18.585461 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e09c8-4a90-446a-9b52-b634c1bf8758 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:18.619779 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce35161-8e2d-4de6-b0a6-52066e55ee1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:18.620345 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce35161-8e2d-4de6-b0a6-52066e55ee1a None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:18.620630 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce35161-8e2d-4de6-b0a6-52066e55ee1a None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:18.620905 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce35161-8e2d-4de6-b0a6-52066e55ee1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:18.630984 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce35161-8e2d-4de6-b0a6-52066e55ee1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:39:18.635963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-dce35161-8e2d-4de6-b0a6-52066e55ee1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:39:18.635963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:18.635963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:18.635963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:18.635963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:18.635963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:18.635963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:18.635963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:18.635963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:18.635963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:18.635963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:18.635963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:18.635963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:39:18.635963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:39:18.635963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:39:18.635963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:39:18.637117 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:39:18.637117 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:39:18.637117 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:39:18.637117 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:39:18.637117 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:18.637117 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:18.637117 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:18.637117 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:18.637117 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:18.637117 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:18.637117 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:18.637117 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:18.637117 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:18.637117 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:18.637117 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:18.637117 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:18.638047 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:18.638047 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:18.638047 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:18.638047 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:18.638047 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:39:18.638047 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:39:18.638047 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:39:18.638047 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:39:18.638047 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:39:18.638047 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:39:18.638047 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:39:18.638047 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:39:18.638047 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 631/1249] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:39:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:39:18.643400 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4011aac-d220-4caa-8fd8-0044a3e396d9 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:18.741557 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4011aac-d220-4caa-8fd8-0044a3e396d9 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:18.769841 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:18.774726 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c33d09-d635-4097-840e-31b0364d698b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:18.775367 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c33d09-d635-4097-840e-31b0364d698b None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:18.775647 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c33d09-d635-4097-840e-31b0364d698b None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:18.775923 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c33d09-d635-4097-840e-31b0364d698b None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:18.776631 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c33d09-d635-4097-840e-31b0364d698b None admin] RBAC: Adding query filter params (name=admin) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:18.776937 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c33d09-d635-4097-840e-31b0364d698b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:18.778297 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c33d09-d635-4097-840e-31b0364d698b None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:18.786842 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 619/1250] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:39:18 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:39:18.792868 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce35161-8e2d-4de6-b0a6-52066e55ee1a None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:18.870395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:18.893735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce35161-8e2d-4de6-b0a6-52066e55ee1a None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:18.930072 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61821244-b792-47f3-97df-b4d4192ae4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:18.933035 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61821244-b792-47f3-97df-b4d4192ae4bc None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:18.933035 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61821244-b792-47f3-97df-b4d4192ae4bc None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:18.933035 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61821244-b792-47f3-97df-b4d4192ae4bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:18.941695 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61821244-b792-47f3-97df-b4d4192ae4bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:18.943221 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61821244-b792-47f3-97df-b4d4192ae4bc None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:18.958813 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-61821244-b792-47f3-97df-b4d4192ae4bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:39:18.958813 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:18.958813 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:18.958813 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:18.958813 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:18.958813 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:18.958813 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:18.958813 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:18.958813 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:18.958813 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:18.958813 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:18.958813 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:18.958813 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:39:18.958813 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:39:18.958813 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:39:18.960002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:39:18.960002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:18.960002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:18.960002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:18.960002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:18.960002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:18.960002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:18.960002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:18.960002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:18.960002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:18.960002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:18.960002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:18.960002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:18.960002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:18.960002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:18.960002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:18.960002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:18.961148 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:39:18.961148 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:39:18.961148 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:39:18.961148 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:39:18.961148 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 632/1251] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:39:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:39:18.965920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c33d09-d635-4097-840e-31b0364d698b None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:19.065748 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c33d09-d635-4097-840e-31b0364d698b None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:19.100799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eddcf0-50b7-4e4c-a67c-7dd8b6d16f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:19.101407 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eddcf0-50b7-4e4c-a67c-7dd8b6d16f00 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:19.101689 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eddcf0-50b7-4e4c-a67c-7dd8b6d16f00 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:19.101963 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62eddcf0-50b7-4e4c-a67c-7dd8b6d16f00 None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:19.102680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eddcf0-50b7-4e4c-a67c-7dd8b6d16f00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:19.102982 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eddcf0-50b7-4e4c-a67c-7dd8b6d16f00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:19.104380 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62eddcf0-50b7-4e4c-a67c-7dd8b6d16f00 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:19.116023 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 620/1252] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:39:18 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:19.121976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61821244-b792-47f3-97df-b4d4192ae4bc None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:19.222477 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61821244-b792-47f3-97df-b4d4192ae4bc None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:19.248605 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:19.256231 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1370c-dd3e-49ad-adbe-ab645249242d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:19.256863 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1370c-dd3e-49ad-adbe-ab645249242d None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:19.257143 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1370c-dd3e-49ad-adbe-ab645249242d None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:19.257418 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1370c-dd3e-49ad-adbe-ab645249242d None admin] PATH_INFO: `/v3/users/placement` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:19.265534 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a1370c-dd3e-49ad-adbe-ab645249242d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:19.266952 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a1370c-dd3e-49ad-adbe-ab645249242d None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:19.278467 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-50a1370c-dd3e-49ad-adbe-ab645249242d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:39:19.278467 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:19.278467 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:19.278467 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:19.278467 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:19.278467 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:19.278467 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:19.278467 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:19.278467 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:19.278467 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:19.278467 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:19.278467 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:19.278467 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:39:19.278467 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:39:19.278467 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:39:19.278467 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:39:19.279682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:19.279682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:19.279682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:39:19.279682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:39:19.279682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:39:19.279682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:39:19.279682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:19.279682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:19.279682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:19.279682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:19.279682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:19.279682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:19.279682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:19.279682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:19.279682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:19.279682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:19.279682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:19.279682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:19.280882 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:19.280882 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:19.280882 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:39:19.280882 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:39:19.280882 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:39:19.280882 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:39:19.280882 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:39:19.280882 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:39:19.280882 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:39:19.280882 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:39:19.280882 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 633/1253] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:39:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:39:19.285743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eddcf0-50b7-4e4c-a67c-7dd8b6d16f00 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:19.342182 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:19.385864 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62eddcf0-50b7-4e4c-a67c-7dd8b6d16f00 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:19.419954 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da84b218-21c3-4f93-bb2c-c2a7e0fbe83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:19.426452 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da84b218-21c3-4f93-bb2c-c2a7e0fbe83e None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:19.426452 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da84b218-21c3-4f93-bb2c-c2a7e0fbe83e None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:19.426452 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da84b218-21c3-4f93-bb2c-c2a7e0fbe83e None admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:19.426452 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da84b218-21c3-4f93-bb2c-c2a7e0fbe83e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:19.426452 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da84b218-21c3-4f93-bb2c-c2a7e0fbe83e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:19.426452 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da84b218-21c3-4f93-bb2c-c2a7e0fbe83e None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:19.442327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 621/1254] 10.222.0.115 () {56 vars in 1148 bytes} [Tue Aug 2 10:39:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:19.448167 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1370c-dd3e-49ad-adbe-ab645249242d None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:19.550939 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1370c-dd3e-49ad-adbe-ab645249242d None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:19.585045 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56197e33-6bc7-4708-afa4-882eff4d56bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:19.585720 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56197e33-6bc7-4708-afa4-882eff4d56bb None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:19.586042 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56197e33-6bc7-4708-afa4-882eff4d56bb None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:19.586368 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56197e33-6bc7-4708-afa4-882eff4d56bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:19.641275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:19.654635 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56197e33-6bc7-4708-afa4-882eff4d56bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:19.656123 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56197e33-6bc7-4708-afa4-882eff4d56bb None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:19.757669 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 634/1255] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:39:19 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 02 10:39:19.763616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da84b218-21c3-4f93-bb2c-c2a7e0fbe83e None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:19.865078 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da84b218-21c3-4f93-bb2c-c2a7e0fbe83e None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:19.881783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:19.902698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b86771-e95d-461d-8bc7-07de270508a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:19.903294 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b86771-e95d-461d-8bc7-07de270508a1 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:19.903577 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b86771-e95d-461d-8bc7-07de270508a1 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:19.903845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b86771-e95d-461d-8bc7-07de270508a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:19.913713 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b86771-e95d-461d-8bc7-07de270508a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:19.915179 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b86771-e95d-461d-8bc7-07de270508a1 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:19.935325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-56b86771-e95d-461d-8bc7-07de270508a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:39:19.935325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:19.935325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:19.935325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:19.935325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:19.935325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:19.935325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:19.935325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:19.935325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:19.935325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:19.935325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:19.935325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:19.935325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:39:19.935325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:39:19.935325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:39:19.936485 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:39:19.936485 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:19.936485 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:19.936485 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:19.936485 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:19.936485 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:19.936485 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:19.936485 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:19.936485 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:19.936485 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:19.936485 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:19.936485 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:19.936485 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:19.936485 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:19.936485 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:19.936485 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:19.936485 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:19.937463 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:39:19.937463 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:39:19.937463 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:39:19.937463 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:39:19.937463 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 622/1256] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:39:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:39:19.943327 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56197e33-6bc7-4708-afa4-882eff4d56bb None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:20.033365 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:20.045322 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56197e33-6bc7-4708-afa4-882eff4d56bb None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:20.079767 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2bf0a2-23d8-4896-b8ee-65ba1cb20ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:20.080440 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2bf0a2-23d8-4896-b8ee-65ba1cb20ffb None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:20.080726 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2bf0a2-23d8-4896-b8ee-65ba1cb20ffb None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:20.081004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2bf0a2-23d8-4896-b8ee-65ba1cb20ffb None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:20.081693 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2bf0a2-23d8-4896-b8ee-65ba1cb20ffb None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:20.082050 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2bf0a2-23d8-4896-b8ee-65ba1cb20ffb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:20.083515 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2bf0a2-23d8-4896-b8ee-65ba1cb20ffb None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:20.094894 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 635/1257] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:39:19 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:20.101425 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b86771-e95d-461d-8bc7-07de270508a1 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:20.203775 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b86771-e95d-461d-8bc7-07de270508a1 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:20.240706 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f6a86-64f7-413f-8a1e-45e20b46d471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:20.241439 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f6a86-64f7-413f-8a1e-45e20b46d471 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:20.241701 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f6a86-64f7-413f-8a1e-45e20b46d471 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:20.241981 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f6a86-64f7-413f-8a1e-45e20b46d471 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:20.248986 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f6a86-64f7-413f-8a1e-45e20b46d471 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:20.250466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f6a86-64f7-413f-8a1e-45e20b46d471 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:20.262237 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-6d6f6a86-64f7-413f-8a1e-45e20b46d471 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:39:20.262237 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:20.262237 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:20.262237 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:20.262237 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:20.262237 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:20.262237 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:20.262237 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:20.262237 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:20.262237 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:20.262237 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:20.262237 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:20.262237 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:39:20.262237 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:39:20.262237 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:39:20.264122 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:39:20.264122 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:20.264122 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:20.264122 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:20.264122 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:20.264122 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:20.264122 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:20.264122 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:20.264122 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:20.264122 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:20.264122 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:20.264122 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:20.264122 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:20.264122 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:20.264122 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:20.264122 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:20.264122 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:20.265411 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:39:20.265411 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:39:20.265411 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:39:20.265411 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:39:20.265411 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:39:20.265411 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:39:20.265411 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:39:20.265411 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:39:20.265411 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 623/1258] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:39:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:39:20.270391 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2bf0a2-23d8-4896-b8ee-65ba1cb20ffb None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:20.376396 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2bf0a2-23d8-4896-b8ee-65ba1cb20ffb None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:20.410689 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c4764-0623-4f2d-9b90-8d91a3dba657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:20.411419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c4764-0623-4f2d-9b90-8d91a3dba657 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:20.411707 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c4764-0623-4f2d-9b90-8d91a3dba657 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:20.411980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c4764-0623-4f2d-9b90-8d91a3dba657 None admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:20.412749 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c4764-0623-4f2d-9b90-8d91a3dba657 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:20.413058 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c4764-0623-4f2d-9b90-8d91a3dba657 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:20.414541 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c4764-0623-4f2d-9b90-8d91a3dba657 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:20.427758 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 636/1259] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:39:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:39:20.433966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f6a86-64f7-413f-8a1e-45e20b46d471 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:20.461117 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:20.534494 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f6a86-64f7-413f-8a1e-45e20b46d471 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:20.568938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f60b6a8-0256-4b5d-8f99-75ce7beb9b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:20.569543 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f60b6a8-0256-4b5d-8f99-75ce7beb9b6a None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:20.569817 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f60b6a8-0256-4b5d-8f99-75ce7beb9b6a None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:20.570082 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f60b6a8-0256-4b5d-8f99-75ce7beb9b6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:20.571040 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f60b6a8-0256-4b5d-8f99-75ce7beb9b6a None admin] RBAC: Adding query filter params (role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac, scope.project.id=c1eae881e5714bf4bd223809e1a2d920, user.id=cbc4b962e2a742b08beef7187cc527e5) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:20.571370 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f60b6a8-0256-4b5d-8f99-75ce7beb9b6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:20.572805 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f60b6a8-0256-4b5d-8f99-75ce7beb9b6a None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:20.578289 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 624/1260] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:39:20 2022] GET /identity/v3/role_assignments?user.id=cbc4b962e2a742b08beef7187cc527e5&scope.project.id=c1eae881e5714bf4bd223809e1a2d920&role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 260 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:39:22.584114 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b96fa5-9054-4fc7-bf34-736069651630 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:22.594344 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b96fa5-9054-4fc7-bf34-736069651630 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:22.594344 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b96fa5-9054-4fc7-bf34-736069651630 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:22.594344 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 637/1261] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:39:22 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:39:22.594653 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb96455-a0b1-4fb7-b67d-508c59a384b7 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:22.594653 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb96455-a0b1-4fb7-b67d-508c59a384b7 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:22.594653 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb96455-a0b1-4fb7-b67d-508c59a384b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:22.620767 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-ddb96455-a0b1-4fb7-b67d-508c59a384b7 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:39:22.622012 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-ddb96455-a0b1-4fb7-b67d-508c59a384b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:39:22.696341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 625/1262] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:39:22 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 02 10:39:22.720147 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b96fa5-9054-4fc7-bf34-736069651630 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:22.732272 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:22.833730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b96fa5-9054-4fc7-bf34-736069651630 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:22.869517 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338dde8-81da-4e45-8c2d-df0e1740f7b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:22.870154 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338dde8-81da-4e45-8c2d-df0e1740f7b3 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:22.870432 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338dde8-81da-4e45-8c2d-df0e1740f7b3 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:22.870704 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6338dde8-81da-4e45-8c2d-df0e1740f7b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:22.880872 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6338dde8-81da-4e45-8c2d-df0e1740f7b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:39:22.885789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-6338dde8-81da-4e45-8c2d-df0e1740f7b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:39:22.885789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:22.885789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:22.885789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:22.885789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:22.885789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:22.885789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:22.885789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:22.885789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:22.885789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:22.885789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:22.885789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:22.885789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:39:22.885789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:39:22.885789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:39:22.885789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:39:22.887149 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:39:22.887149 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:39:22.887149 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:39:22.887149 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:39:22.887149 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:22.887149 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:22.887149 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:22.887149 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:22.887149 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:22.887149 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:22.887149 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:22.887149 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:22.887149 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:22.887149 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:22.887149 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:22.887149 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:22.888184 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:22.888184 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:22.888184 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:22.888184 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:22.888184 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:39:22.888184 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:39:22.888184 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:39:22.888184 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:39:22.888184 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:39:22.888184 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:39:22.888184 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:39:22.888184 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:39:22.888184 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 638/1263] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:39:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:39:22.893134 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb96455-a0b1-4fb7-b67d-508c59a384b7 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:22.912901 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:22.993629 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb96455-a0b1-4fb7-b67d-508c59a384b7 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:23.029134 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b027dc9-d621-45ed-90f0-39c9f7bd5b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:23.029740 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b027dc9-d621-45ed-90f0-39c9f7bd5b85 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:23.030017 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b027dc9-d621-45ed-90f0-39c9f7bd5b85 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:23.030286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b027dc9-d621-45ed-90f0-39c9f7bd5b85 None admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:23.031281 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b027dc9-d621-45ed-90f0-39c9f7bd5b85 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:23.031408 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b027dc9-d621-45ed-90f0-39c9f7bd5b85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:23.032688 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b027dc9-d621-45ed-90f0-39c9f7bd5b85 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:23.042236 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 626/1264] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:39:22 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:39:23.048248 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338dde8-81da-4e45-8c2d-df0e1740f7b3 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:23.117953 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:23.153848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6338dde8-81da-4e45-8c2d-df0e1740f7b3 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:23.190551 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d474c9d-2433-48ab-ae00-3a7a062b9731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:23.191747 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d474c9d-2433-48ab-ae00-3a7a062b9731 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:23.192057 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d474c9d-2433-48ab-ae00-3a7a062b9731 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:23.192332 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d474c9d-2433-48ab-ae00-3a7a062b9731 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:23.203618 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d474c9d-2433-48ab-ae00-3a7a062b9731 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:23.213673 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d474c9d-2433-48ab-ae00-3a7a062b9731 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:23.220422 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-8d474c9d-2433-48ab-ae00-3a7a062b9731 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:39:23.220422 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:23.220422 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:23.220422 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:23.220422 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:23.220422 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:23.220422 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:23.220422 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:23.220422 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:23.220422 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:23.220422 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:23.220422 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:23.220422 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:39:23.220422 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:39:23.220422 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:39:23.221521 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:39:23.221521 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:23.221521 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:23.221521 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:23.221521 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:23.221521 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:23.221521 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:23.221521 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:23.221521 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:23.221521 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:23.221521 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:23.221521 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:23.221521 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:23.221521 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:23.221521 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:23.221521 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:23.221521 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:23.222840 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:39:23.222840 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:39:23.222840 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:39:23.222840 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:39:23.222840 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 639/1265] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:39:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:39:23.227702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b027dc9-d621-45ed-90f0-39c9f7bd5b85 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:23.311195 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:23.332447 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b027dc9-d621-45ed-90f0-39c9f7bd5b85 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:23.366833 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab820c9-cc38-4cb1-8387-afd7c6b051a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:23.367518 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab820c9-cc38-4cb1-8387-afd7c6b051a5 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:23.367799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab820c9-cc38-4cb1-8387-afd7c6b051a5 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:23.368072 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab820c9-cc38-4cb1-8387-afd7c6b051a5 None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:23.368763 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab820c9-cc38-4cb1-8387-afd7c6b051a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:23.369069 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab820c9-cc38-4cb1-8387-afd7c6b051a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:23.370440 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab820c9-cc38-4cb1-8387-afd7c6b051a5 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:23.382932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 627/1266] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:39:23 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:23.388914 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d474c9d-2433-48ab-ae00-3a7a062b9731 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:23.494614 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d474c9d-2433-48ab-ae00-3a7a062b9731 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:23.515204 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:23.529341 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d5060-6a2e-4f69-9064-716299b1f896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:23.529979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d5060-6a2e-4f69-9064-716299b1f896 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:23.530259 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d5060-6a2e-4f69-9064-716299b1f896 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:23.530552 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1d5060-6a2e-4f69-9064-716299b1f896 None admin] PATH_INFO: `/v3/users/placement` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:23.538645 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d5060-6a2e-4f69-9064-716299b1f896 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:23.540113 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1d5060-6a2e-4f69-9064-716299b1f896 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:23.552040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-3d1d5060-6a2e-4f69-9064-716299b1f896 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:39:23.552040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:23.552040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:23.552040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:23.552040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:23.552040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:23.552040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:23.552040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:23.552040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:23.552040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:23.552040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:23.552040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:23.552040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:39:23.552040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:39:23.552040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:39:23.552040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:39:23.553307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:23.553307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:23.553307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:39:23.553307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:39:23.553307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:39:23.553307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:39:23.553307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:23.553307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:23.553307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:23.553307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:23.553307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:23.553307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:23.553307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:23.553307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:23.553307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:23.553307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:23.553307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:23.553307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:23.554392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:23.554392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:23.554392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:39:23.554392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:39:23.554392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:39:23.554392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:39:23.554392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:39:23.554392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:39:23.554392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:39:23.554392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:39:23.554392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 640/1267] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:39:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:39:23.559030 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab820c9-cc38-4cb1-8387-afd7c6b051a5 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:23.660309 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab820c9-cc38-4cb1-8387-afd7c6b051a5 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:23.696272 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a2b38-91f3-4b80-93df-be0c6b08edb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:23.696886 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a2b38-91f3-4b80-93df-be0c6b08edb3 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:23.697159 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a2b38-91f3-4b80-93df-be0c6b08edb3 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:23.697426 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09a2b38-91f3-4b80-93df-be0c6b08edb3 None admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:23.698463 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09a2b38-91f3-4b80-93df-be0c6b08edb3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:23.698765 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09a2b38-91f3-4b80-93df-be0c6b08edb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:23.700216 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09a2b38-91f3-4b80-93df-be0c6b08edb3 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:23.710897 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:23.719788 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 628/1268] 10.222.0.115 () {56 vars in 1148 bytes} [Tue Aug 2 10:39:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:23.727075 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d5060-6a2e-4f69-9064-716299b1f896 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:23.834726 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1d5060-6a2e-4f69-9064-716299b1f896 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:23.870509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae2b40-bcde-4a51-a9a1-d12e2703417e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:23.871163 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae2b40-bcde-4a51-a9a1-d12e2703417e None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:23.871443 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae2b40-bcde-4a51-a9a1-d12e2703417e None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:23.871712 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ae2b40-bcde-4a51-a9a1-d12e2703417e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:23.885314 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ae2b40-bcde-4a51-a9a1-d12e2703417e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:23.886751 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ae2b40-bcde-4a51-a9a1-d12e2703417e None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:23.902565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-d2ae2b40-bcde-4a51-a9a1-d12e2703417e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:39:23.902565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:23.902565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:23.902565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:23.902565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:23.902565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:23.902565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:23.902565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:23.902565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:23.902565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:23.902565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:23.902565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:23.902565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:39:23.902565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:39:23.902565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:39:23.903825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:39:23.903825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:23.903825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:23.903825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:23.903825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:23.903825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:23.903825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:23.903825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:23.903825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:23.903825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:23.903825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:23.903825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:23.903825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:23.903825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:23.903825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:23.903825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:23.903825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:23.904900 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:39:23.904900 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:39:23.904900 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:39:23.904900 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:39:23.904900 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 641/1269] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:39:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:39:23.909877 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a2b38-91f3-4b80-93df-be0c6b08edb3 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:24.016792 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09a2b38-91f3-4b80-93df-be0c6b08edb3 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:24.052781 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e38d82-bad5-47ea-86f4-76c7e62a248b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:24.053396 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e38d82-bad5-47ea-86f4-76c7e62a248b None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:24.053671 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e38d82-bad5-47ea-86f4-76c7e62a248b None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:24.053946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e38d82-bad5-47ea-86f4-76c7e62a248b None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:24.054628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e38d82-bad5-47ea-86f4-76c7e62a248b None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:24.054932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e38d82-bad5-47ea-86f4-76c7e62a248b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:24.056357 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e38d82-bad5-47ea-86f4-76c7e62a248b None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:24.068878 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 629/1270] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:39:23 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:24.075220 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae2b40-bcde-4a51-a9a1-d12e2703417e None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:24.108307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:24.181631 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ae2b40-bcde-4a51-a9a1-d12e2703417e None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:24.216819 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6feb4e4-8f15-4f37-b929-88d3f316b600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:24.217505 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6feb4e4-8f15-4f37-b929-88d3f316b600 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:24.217827 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6feb4e4-8f15-4f37-b929-88d3f316b600 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:24.218155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6feb4e4-8f15-4f37-b929-88d3f316b600 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:24.225947 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6feb4e4-8f15-4f37-b929-88d3f316b600 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:24.227412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6feb4e4-8f15-4f37-b929-88d3f316b600 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:24.239769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-a6feb4e4-8f15-4f37-b929-88d3f316b600 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:39:24.239769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:24.239769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:24.239769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:24.239769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:24.239769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:24.239769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:24.239769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:24.239769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:24.239769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:24.239769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:24.239769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:24.239769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:39:24.239769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:39:24.239769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:39:24.240930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:39:24.240930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:24.240930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:24.240930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:24.240930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:24.240930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:24.240930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:24.240930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:24.240930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:24.240930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:24.240930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:24.240930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:24.240930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:24.240930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:24.240930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:24.240930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:24.240930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:24.242280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:39:24.242280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:39:24.242280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:39:24.242280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:39:24.242280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:39:24.242280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:39:24.242280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:39:24.242280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:39:24.243100 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 642/1271] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:39:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:39:24.246932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e38d82-bad5-47ea-86f4-76c7e62a248b None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:24.310550 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:24.357205 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e38d82-bad5-47ea-86f4-76c7e62a248b None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:24.393142 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e849902b-7f54-4be7-882c-e562bb280e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:24.393750 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849902b-7f54-4be7-882c-e562bb280e9d None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:24.394030 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849902b-7f54-4be7-882c-e562bb280e9d None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:24.394418 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e849902b-7f54-4be7-882c-e562bb280e9d None admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:24.395389 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e849902b-7f54-4be7-882c-e562bb280e9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:24.395705 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e849902b-7f54-4be7-882c-e562bb280e9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:24.397135 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e849902b-7f54-4be7-882c-e562bb280e9d None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:24.409096 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 630/1272] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:39:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:24.415408 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6feb4e4-8f15-4f37-b929-88d3f316b600 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:24.511498 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:24.520855 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6feb4e4-8f15-4f37-b929-88d3f316b600 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:24.556832 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9178a882-32ab-44a9-b1b0-ee56560b692c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:24.557610 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178a882-32ab-44a9-b1b0-ee56560b692c None admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:24.557898 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178a882-32ab-44a9-b1b0-ee56560b692c None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:24.558174 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9178a882-32ab-44a9-b1b0-ee56560b692c None admin] PATH_INFO: `/v3/projects/c1eae881e5714bf4bd223809e1a2d920/users/cbc4b962e2a742b08beef7187cc527e5/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:24.560254 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9178a882-32ab-44a9-b1b0-ee56560b692c None admin] RBAC: Authorizing `identity:create_grant(project_id=c1eae881e5714bf4bd223809e1a2d920, user_id=cbc4b962e2a742b08beef7187cc527e5, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:24.561852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9178a882-32ab-44a9-b1b0-ee56560b692c None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:24.621868 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 643/1273] 10.222.0.115 () {58 vars in 1444 bytes} [Tue Aug 2 10:39:24 2022] PUT /identity/v3/projects/c1eae881e5714bf4bd223809e1a2d920/users/cbc4b962e2a742b08beef7187cc527e5/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:39:26.626110 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ba4b1-f86d-4c91-9f95-e9439e218b97 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:26.626681 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ba4b1-f86d-4c91-9f95-e9439e218b97 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:26.626681 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ba4b1-f86d-4c91-9f95-e9439e218b97 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:26.628081 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 631/1274] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:39:26 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:39:26.633918 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00afcf-507b-4655-9711-0f7abdd414f8 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:26.634452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00afcf-507b-4655-9711-0f7abdd414f8 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:26.634940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00afcf-507b-4655-9711-0f7abdd414f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:26.654518 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-1a00afcf-507b-4655-9711-0f7abdd414f8 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:39:26.655733 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-1a00afcf-507b-4655-9711-0f7abdd414f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:39:26.723796 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 644/1275] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:39:26 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:39:26.747100 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ba4b1-f86d-4c91-9f95-e9439e218b97 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:26.872002 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ba4b1-f86d-4c91-9f95-e9439e218b97 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:26.889543 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:26.906869 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45089c78-0673-4e2c-8ec9-de842103ce9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:26.907533 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45089c78-0673-4e2c-8ec9-de842103ce9c None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:26.907817 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45089c78-0673-4e2c-8ec9-de842103ce9c None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:26.908074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45089c78-0673-4e2c-8ec9-de842103ce9c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:26.922392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45089c78-0673-4e2c-8ec9-de842103ce9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 02 10:39:26.931371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-45089c78-0673-4e2c-8ec9-de842103ce9c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:39:26.931371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:26.931371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:26.931371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:26.931371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:26.931371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:26.931371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:26.931371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:26.931371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:26.931371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:26.931371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:26.931371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:26.931371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 02 10:39:26.931371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 02 10:39:26.931371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 02 10:39:26.931371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 02 10:39:26.934889 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 02 10:39:26.934889 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 02 10:39:26.934889 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 02 10:39:26.934889 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 02 10:39:26.934889 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:26.934889 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:26.934889 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:26.934889 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:26.934889 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:26.934889 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:26.934889 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:26.934889 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:26.934889 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:26.934889 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:26.934889 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:26.934889 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:26.936008 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:26.936008 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:26.936008 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:26.936008 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:26.936008 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 02 10:39:26.936008 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 02 10:39:26.936008 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 02 10:39:26.936008 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 02 10:39:26.936008 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 02 10:39:26.936008 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 02 10:39:26.936008 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 02 10:39:26.936008 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:39:26.936008 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 632/1276] 10.222.0.115 () {56 vars in 1101 bytes} [Tue Aug 2 10:39:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:39:26.939453 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00afcf-507b-4655-9711-0f7abdd414f8 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:27.047652 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:27.067086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00afcf-507b-4655-9711-0f7abdd414f8 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:27.102114 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215bec3-7907-41a3-8f30-25fe23df41d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:27.102841 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215bec3-7907-41a3-8f30-25fe23df41d7 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:27.103521 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215bec3-7907-41a3-8f30-25fe23df41d7 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:27.103819 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4215bec3-7907-41a3-8f30-25fe23df41d7 None admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:27.104676 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215bec3-7907-41a3-8f30-25fe23df41d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:27.105074 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215bec3-7907-41a3-8f30-25fe23df41d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:27.106675 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4215bec3-7907-41a3-8f30-25fe23df41d7 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:27.115738 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 645/1277] 10.222.0.115 () {56 vars in 1104 bytes} [Tue Aug 2 10:39:26 2022] GET /identity/v3/roles?name=admin => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:27.122153 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45089c78-0673-4e2c-8ec9-de842103ce9c None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:27.233547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45089c78-0673-4e2c-8ec9-de842103ce9c None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:27.273860 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f404a278-eda2-4031-93a3-ebb0009af65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:27.274492 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404a278-eda2-4031-93a3-ebb0009af65e None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:27.274767 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404a278-eda2-4031-93a3-ebb0009af65e None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:27.275033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f404a278-eda2-4031-93a3-ebb0009af65e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:27.286482 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f404a278-eda2-4031-93a3-ebb0009af65e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:27.287996 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f404a278-eda2-4031-93a3-ebb0009af65e None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:27.303939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-f404a278-eda2-4031-93a3-ebb0009af65e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:39:27.303939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:27.303939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:27.303939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:27.303939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:27.303939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:27.303939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:27.303939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:27.303939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:27.303939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:27.303939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:27.303939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:27.303939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:39:27.303939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:39:27.303939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:39:27.305156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:39:27.305156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:27.305156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:27.305156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:27.305156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:27.305156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:27.305156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:27.305156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:27.305156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:27.305156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:27.305156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:27.305156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:27.305156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:27.305156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:27.305156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:27.305156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:27.305156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:27.306161 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:39:27.306161 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:39:27.306161 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:39:27.306161 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:39:27.306161 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 633/1278] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:39:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:39:27.311779 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215bec3-7907-41a3-8f30-25fe23df41d7 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:27.417438 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4215bec3-7907-41a3-8f30-25fe23df41d7 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:27.462877 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25461164-9f9b-4c65-98d4-e8ce605c0d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:27.463535 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25461164-9f9b-4c65-98d4-e8ce605c0d89 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:27.463852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25461164-9f9b-4c65-98d4-e8ce605c0d89 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:27.464129 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25461164-9f9b-4c65-98d4-e8ce605c0d89 None admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:27.464850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25461164-9f9b-4c65-98d4-e8ce605c0d89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:27.465154 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25461164-9f9b-4c65-98d4-e8ce605c0d89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:27.466517 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25461164-9f9b-4c65-98d4-e8ce605c0d89 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:27.478327 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 646/1279] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:39:27 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:27.484691 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f404a278-eda2-4031-93a3-ebb0009af65e None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:27.517269 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:27.586219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f404a278-eda2-4031-93a3-ebb0009af65e None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:27.621037 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3384239-3482-44b6-8e1d-e45187102cf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:27.621671 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3384239-3482-44b6-8e1d-e45187102cf0 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:27.621950 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3384239-3482-44b6-8e1d-e45187102cf0 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:27.622224 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3384239-3482-44b6-8e1d-e45187102cf0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:27.636041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3384239-3482-44b6-8e1d-e45187102cf0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:27.637502 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3384239-3482-44b6-8e1d-e45187102cf0 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:27.650523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-d3384239-3482-44b6-8e1d-e45187102cf0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:39:27.650523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:27.650523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:27.650523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:27.650523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:27.650523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:27.650523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:27.650523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:27.650523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:27.650523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:27.650523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:27.650523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:27.650523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 02 10:39:27.650523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 02 10:39:27.650523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 02 10:39:27.650523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 02 10:39:27.651780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:27.651780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:27.651780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 02 10:39:27.651780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:39:27.651780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 02 10:39:27.651780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 02 10:39:27.651780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:27.651780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:27.651780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:27.651780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:27.651780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:27.651780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:27.651780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:27.651780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:27.651780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:27.651780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:27.651780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:27.651780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:27.652934 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:27.652934 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:27.652934 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 02 10:39:27.652934 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 02 10:39:27.652934 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 02 10:39:27.652934 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 02 10:39:27.652934 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 02 10:39:27.652934 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 02 10:39:27.652934 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 02 10:39:27.652934 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:39:27.652934 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 634/1280] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:39:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:39:27.657831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25461164-9f9b-4c65-98d4-e8ce605c0d89 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:27.662724 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:27.758767 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25461164-9f9b-4c65-98d4-e8ce605c0d89 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:27.798270 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398bb58f-f3ee-4f58-9451-5a9c8eef4e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:27.798881 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398bb58f-f3ee-4f58-9451-5a9c8eef4e84 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:27.799191 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398bb58f-f3ee-4f58-9451-5a9c8eef4e84 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:27.799462 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398bb58f-f3ee-4f58-9451-5a9c8eef4e84 None admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:27.800292 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398bb58f-f3ee-4f58-9451-5a9c8eef4e84 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:27.801196 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398bb58f-f3ee-4f58-9451-5a9c8eef4e84 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:27.802983 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398bb58f-f3ee-4f58-9451-5a9c8eef4e84 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:27.820693 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 647/1281] 10.222.0.115 () {56 vars in 1148 bytes} [Tue Aug 2 10:39:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 380 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:27.827439 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3384239-3482-44b6-8e1d-e45187102cf0 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:27.924349 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:27.934551 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3384239-3482-44b6-8e1d-e45187102cf0 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:27.971592 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e9d6d-d72b-4940-ad4d-fddb24b11da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:27.972236 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e9d6d-d72b-4940-ad4d-fddb24b11da5 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:27.973665 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e9d6d-d72b-4940-ad4d-fddb24b11da5 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:27.973943 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1e9d6d-d72b-4940-ad4d-fddb24b11da5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:28.047409 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e9d6d-d72b-4940-ad4d-fddb24b11da5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:28.048864 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1e9d6d-d72b-4940-ad4d-fddb24b11da5 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:28.140319 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:28.151715 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 635/1282] 10.222.0.115 () {58 vars in 1287 bytes} [Tue Aug 2 10:39:27 2022] GET /identity/v3/auth/tokens => generated 2123 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 02 10:39:28.157830 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398bb58f-f3ee-4f58-9451-5a9c8eef4e84 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:28.221852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:28.261554 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398bb58f-f3ee-4f58-9451-5a9c8eef4e84 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:28.295712 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f43922-ea2e-4960-8bfc-98b8b135791d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:28.296343 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f43922-ea2e-4960-8bfc-98b8b135791d None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:28.296618 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f43922-ea2e-4960-8bfc-98b8b135791d None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:28.296884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f43922-ea2e-4960-8bfc-98b8b135791d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:28.307777 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f43922-ea2e-4960-8bfc-98b8b135791d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:28.309208 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f43922-ea2e-4960-8bfc-98b8b135791d None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:28.325788 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-e9f43922-ea2e-4960-8bfc-98b8b135791d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:39:28.325788 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:28.325788 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:28.325788 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:28.325788 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:28.325788 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:28.325788 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:28.325788 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:28.325788 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:28.325788 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:28.325788 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:28.325788 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:28.325788 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 02 10:39:28.325788 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 02 10:39:28.325788 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 02 10:39:28.330378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 02 10:39:28.330378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:28.330378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:28.330378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:28.330378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:28.330378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:28.330378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:28.330378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:28.330378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:28.330378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:28.330378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:28.330378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:28.330378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:28.330378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:28.330378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:28.330378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:28.330378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:28.331580 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 02 10:39:28.331580 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 02 10:39:28.331580 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 02 10:39:28.331580 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:39:28.331580 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 648/1283] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:39:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 02 10:39:28.333092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e9d6d-d72b-4940-ad4d-fddb24b11da5 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:28.439215 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1e9d6d-d72b-4940-ad4d-fddb24b11da5 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:28.480145 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9e8ab-cecd-4399-ad2a-5e58ce4755a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:28.480686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9e8ab-cecd-4399-ad2a-5e58ce4755a5 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:28.480960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9e8ab-cecd-4399-ad2a-5e58ce4755a5 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:28.481227 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b9e8ab-cecd-4399-ad2a-5e58ce4755a5 None admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:28.481910 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9e8ab-cecd-4399-ad2a-5e58ce4755a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:28.482207 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9e8ab-cecd-4399-ad2a-5e58ce4755a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:28.483604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b9e8ab-cecd-4399-ad2a-5e58ce4755a5 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:28.496070 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 636/1284] 10.222.0.115 () {56 vars in 1114 bytes} [Tue Aug 2 10:39:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:39:28.501827 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f43922-ea2e-4960-8bfc-98b8b135791d None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:28.612228 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f43922-ea2e-4960-8bfc-98b8b135791d None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:28.626565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:28.646677 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b81adb-ddaa-4b35-b526-a856c9a026b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:28.647096 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b81adb-ddaa-4b35-b526-a856c9a026b5 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:28.647425 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b81adb-ddaa-4b35-b526-a856c9a026b5 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:28.647780 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b81adb-ddaa-4b35-b526-a856c9a026b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:28.654879 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b81adb-ddaa-4b35-b526-a856c9a026b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:28.656356 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b81adb-ddaa-4b35-b526-a856c9a026b5 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:28.668346 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application [None req-b5b81adb-ddaa-4b35-b526-a856c9a026b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:39:28.668346 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:28.668346 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:28.668346 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:28.668346 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:28.668346 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:28.668346 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:28.668346 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:28.668346 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:28.668346 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:28.668346 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:28.668346 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:28.668346 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 02 10:39:28.668346 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 02 10:39:28.668346 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 02 10:39:28.674854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 02 10:39:28.674854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:28.674854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:28.674854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:28.674854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:28.674854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:28.674854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:28.674854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:28.674854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:28.674854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:28.674854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:28.674854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:28.674854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:28.674854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:28.674854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:28.674854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:28.674854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:28.676744 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 02 10:39:28.676744 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 02 10:39:28.676744 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 02 10:39:28.676744 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 02 10:39:28.676744 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 02 10:39:28.676744 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 02 10:39:28.676744 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 02 10:39:28.676744 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: ERROR keystone.server.flask.application Aug 02 10:39:28.676744 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 649/1285] 10.222.0.115 () {56 vars in 1116 bytes} [Tue Aug 2 10:39:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:39:28.677416 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9e8ab-cecd-4399-ad2a-5e58ce4755a5 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:28.731505 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:28.782415 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b9e8ab-cecd-4399-ad2a-5e58ce4755a5 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:28.817692 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572edd4d-ea77-4ea6-a8ce-86e618886a39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:28.818285 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572edd4d-ea77-4ea6-a8ce-86e618886a39 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:28.818576 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572edd4d-ea77-4ea6-a8ce-86e618886a39 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:28.818850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572edd4d-ea77-4ea6-a8ce-86e618886a39 None admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:28.819632 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572edd4d-ea77-4ea6-a8ce-86e618886a39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:28.819939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572edd4d-ea77-4ea6-a8ce-86e618886a39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:28.821438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572edd4d-ea77-4ea6-a8ce-86e618886a39 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:28.834277 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 637/1286] 10.222.0.115 () {56 vars in 1153 bytes} [Tue Aug 2 10:39:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 431 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:28.844016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b81adb-ddaa-4b35-b526-a856c9a026b5 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:28.950194 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b81adb-ddaa-4b35-b526-a856c9a026b5 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:28.987092 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e9c011-7c32-4648-938e-51655616491e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:28.987630 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9c011-7c32-4648-938e-51655616491e None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:28.988041 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9c011-7c32-4648-938e-51655616491e None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:28.988313 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e9c011-7c32-4648-938e-51655616491e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:28.989094 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e9c011-7c32-4648-938e-51655616491e None admin] RBAC: Adding query filter params (role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac, scope.project.id=c1eae881e5714bf4bd223809e1a2d920, user.id=cbc4b962e2a742b08beef7187cc527e5) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:28.989399 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e9c011-7c32-4648-938e-51655616491e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:28.990826 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e9c011-7c32-4648-938e-51655616491e None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:29.000910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 650/1287] 10.222.0.115 () {56 vars in 1379 bytes} [Tue Aug 2 10:39:28 2022] GET /identity/v3/role_assignments?user.id=cbc4b962e2a742b08beef7187cc527e5&scope.project.id=c1eae881e5714bf4bd223809e1a2d920&role.id=d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 610 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:31.059885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345b2e6-2ae6-451f-bf8e-663f66ef8452 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:31.060452 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345b2e6-2ae6-451f-bf8e-663f66ef8452 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:31.060452 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5345b2e6-2ae6-451f-bf8e-663f66ef8452 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:31.061771 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 638/1288] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:39:31 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:39:31.067145 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd8313-f8a4-435e-8a64-ab78a5e7478f None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:31.069580 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd8313-f8a4-435e-8a64-ab78a5e7478f None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:31.069580 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd8313-f8a4-435e-8a64-ab78a5e7478f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:31.086669 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-93fd8313-f8a4-435e-8a64-ab78a5e7478f None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:39:31.087889 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-93fd8313-f8a4-435e-8a64-ab78a5e7478f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:39:31.118871 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:31.156230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 651/1289] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:39:31 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:39:31.179642 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345b2e6-2ae6-451f-bf8e-663f66ef8452 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:31.301977 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5345b2e6-2ae6-451f-bf8e-663f66ef8452 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:31.316295 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:31.340541 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb4bbc-9833-4d6c-8843-f22904436c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:31.341305 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb4bbc-9833-4d6c-8843-f22904436c17 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:31.341598 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb4bbc-9833-4d6c-8843-f22904436c17 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:31.341873 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fb4bbc-9833-4d6c-8843-f22904436c17 None admin] PATH_INFO: `/v3/services/placement` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:31.342501 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb4bbc-9833-4d6c-8843-f22904436c17 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:31.344047 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fb4bbc-9833-4d6c-8843-f22904436c17 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:31.354850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-f1fb4bbc-9833-4d6c-8843-f22904436c17 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 10:39:31.354850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:31.354850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:39:31.354850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:39:31.354850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:39:31.354850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:39:31.354850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:39:31.354850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:39:31.354850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 10:39:31.354850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:39:31.354850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:39:31.354850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:39:31.354850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:31.354850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:31.354850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:31.354850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:31.354850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:31.356948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:31.356948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:31.356948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:31.356948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:31.356948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:31.356948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:31.356948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:39:31.356948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:39:31.356948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:39:31.356948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:39:31.356948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:31.356948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:31.356948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:31.356948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:31.356948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:31.356948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:31.358968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:31.358968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:31.358968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:31.358968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:31.358968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:31.358968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:31.358968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:31.358968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:31.358968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:31.358968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:31.358968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:39:31.358968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:39:31.358968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 10:39:31.358968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:39:31.358968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 639/1290] 10.222.0.115 () {56 vars in 1122 bytes} [Tue Aug 2 10:39:31 2022] GET /identity/v3/services/placement => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:39:31.365048 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd8313-f8a4-435e-8a64-ab78a5e7478f None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:31.476559 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd8313-f8a4-435e-8a64-ab78a5e7478f None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:31.516145 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcd1a10-d2a6-4af2-8bfb-30683db23442 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:31.516767 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd1a10-d2a6-4af2-8bfb-30683db23442 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:31.517055 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd1a10-d2a6-4af2-8bfb-30683db23442 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:31.517329 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcd1a10-d2a6-4af2-8bfb-30683db23442 None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:31.518095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd1a10-d2a6-4af2-8bfb-30683db23442 None admin] RBAC: Adding query filter params (name=placement) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:31.518315 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd1a10-d2a6-4af2-8bfb-30683db23442 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:31.520357 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcd1a10-d2a6-4af2-8bfb-30683db23442 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:31.525721 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 652/1291] 10.222.0.115 () {56 vars in 1121 bytes} [Tue Aug 2 10:39:31 2022] GET /identity/v3/services?name=placement => generated 127 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:31.535738 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb4bbc-9833-4d6c-8843-f22904436c17 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:31.644665 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fb4bbc-9833-4d6c-8843-f22904436c17 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:31.686315 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0182d37b-46a7-4d68-9ac4-beed37cea971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:31.686936 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182d37b-46a7-4d68-9ac4-beed37cea971 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:31.687252 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182d37b-46a7-4d68-9ac4-beed37cea971 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:31.687528 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0182d37b-46a7-4d68-9ac4-beed37cea971 None admin] PATH_INFO: `/v3/services` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:31.688258 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0182d37b-46a7-4d68-9ac4-beed37cea971 None admin] RBAC: Adding query filter params (type=placement) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:31.688557 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0182d37b-46a7-4d68-9ac4-beed37cea971 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:31.689964 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0182d37b-46a7-4d68-9ac4-beed37cea971 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:31.694833 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 640/1292] 10.222.0.115 () {56 vars in 1121 bytes} [Tue Aug 2 10:39:31 2022] GET /identity/v3/services?type=placement => generated 127 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:33.778001 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedca99-0e36-401f-a2be-30f2e11d8ce2 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:33.782425 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedca99-0e36-401f-a2be-30f2e11d8ce2 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:33.782425 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eedca99-0e36-401f-a2be-30f2e11d8ce2 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:33.782425 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 653/1293] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:39:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:39:33.785299 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ee97a-8698-4146-af58-0d2c1b61b737 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:33.785741 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ee97a-8698-4146-af58-0d2c1b61b737 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:33.785848 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ee97a-8698-4146-af58-0d2c1b61b737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:33.804051 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-122ee97a-8698-4146-af58-0d2c1b61b737 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:39:33.805263 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-122ee97a-8698-4146-af58-0d2c1b61b737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:39:33.836348 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:33.873175 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 641/1294] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:39:33 2022] POST /identity/v3/auth/tokens => generated 2123 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:39:33.897857 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eedca99-0e36-401f-a2be-30f2e11d8ce2 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:33.929439 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:34.006928 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eedca99-0e36-401f-a2be-30f2e11d8ce2 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:34.043221 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a73c5c-4d53-4f2b-ba2f-d5dc65912064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:34.043766 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a73c5c-4d53-4f2b-ba2f-d5dc65912064 None admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:34.044045 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a73c5c-4d53-4f2b-ba2f-d5dc65912064 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:34.044314 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a73c5c-4d53-4f2b-ba2f-d5dc65912064 None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:34.045008 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a73c5c-4d53-4f2b-ba2f-d5dc65912064 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:34.046348 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a73c5c-4d53-4f2b-ba2f-d5dc65912064 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:34.152400 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 654/1295] 10.222.0.115 () {60 vars in 1146 bytes} [Tue Aug 2 10:39:33 2022] POST /identity/v3/services => generated 247 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:39:36.192882 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca849b-8045-44f9-965e-a5da746ef1e7 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:36.193426 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca849b-8045-44f9-965e-a5da746ef1e7 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:36.193426 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ca849b-8045-44f9-965e-a5da746ef1e7 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:36.194785 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 642/1296] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:39:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:39:36.200146 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf8019f-b39f-417e-adb7-beb1243cc622 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:36.200571 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf8019f-b39f-417e-adb7-beb1243cc622 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:36.200684 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf8019f-b39f-417e-adb7-beb1243cc622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:36.218914 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-1bf8019f-b39f-417e-adb7-beb1243cc622 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:39:36.220640 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-1bf8019f-b39f-417e-adb7-beb1243cc622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:39:36.294979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 655/1297] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:39:36 2022] POST /identity/v3/auth/tokens => generated 2226 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:39:36.317326 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca849b-8045-44f9-965e-a5da746ef1e7 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:36.421137 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ca849b-8045-44f9-965e-a5da746ef1e7 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:36.456682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c08ccb-8863-40b8-a366-2e690e1c7264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:36.457309 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c08ccb-8863-40b8-a366-2e690e1c7264 None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:36.457589 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c08ccb-8863-40b8-a366-2e690e1c7264 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:36.457867 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c08ccb-8863-40b8-a366-2e690e1c7264 None admin] PATH_INFO: `/v3/services/placement` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:36.458499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c08ccb-8863-40b8-a366-2e690e1c7264 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:36.459944 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c08ccb-8863-40b8-a366-2e690e1c7264 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:36.469223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-c4c08ccb-8863-40b8-a366-2e690e1c7264 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 10:39:36.469223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:36.469223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:39:36.469223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:39:36.469223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:39:36.469223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:39:36.469223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:39:36.469223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:39:36.469223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 10:39:36.469223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:39:36.469223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:39:36.469223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:39:36.469223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:36.469223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:36.469223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:36.469223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:36.469223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:36.470437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:36.470437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:36.470437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:36.470437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:36.470437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:36.470437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:36.470437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:39:36.470437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:39:36.470437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:39:36.470437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:39:36.470437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:36.470437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:36.470437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:36.470437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:36.470437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:36.470437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:36.471468 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:36.471468 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:36.471468 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:36.471468 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:36.471468 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:36.471468 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:36.471468 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:36.471468 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:36.471468 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:36.471468 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:36.471468 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:39:36.471468 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:39:36.471468 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 10:39:36.471468 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:39:36.471468 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 643/1298] 10.222.0.115 () {56 vars in 1122 bytes} [Tue Aug 2 10:39:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:39:36.476581 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf8019f-b39f-417e-adb7-beb1243cc622 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:36.495157 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:36.577052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf8019f-b39f-417e-adb7-beb1243cc622 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:36.611444 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf6670-018f-4f21-af7c-c1ecb5b01713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:36.612077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf6670-018f-4f21-af7c-c1ecb5b01713 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:36.612932 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf6670-018f-4f21-af7c-c1ecb5b01713 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:36.613386 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbf6670-018f-4f21-af7c-c1ecb5b01713 None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:36.614199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf6670-018f-4f21-af7c-c1ecb5b01713 None admin] RBAC: Adding query filter params (name=placement) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:36.614501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf6670-018f-4f21-af7c-c1ecb5b01713 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:36.615888 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbf6670-018f-4f21-af7c-c1ecb5b01713 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:36.621028 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 656/1299] 10.222.0.115 () {56 vars in 1121 bytes} [Tue Aug 2 10:39:36 2022] GET /identity/v3/services?name=placement => generated 360 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:36.627125 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c08ccb-8863-40b8-a366-2e690e1c7264 None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:36.658072 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:36.743114 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c08ccb-8863-40b8-a366-2e690e1c7264 None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:36.776434 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a154403-b3f5-4c2d-8237-7e480a26c52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:36.777043 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a154403-b3f5-4c2d-8237-7e480a26c52e None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:36.777316 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a154403-b3f5-4c2d-8237-7e480a26c52e None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:36.777581 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a154403-b3f5-4c2d-8237-7e480a26c52e None admin] PATH_INFO: `/v3/endpoints` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:36.778324 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a154403-b3f5-4c2d-8237-7e480a26c52e None admin] RBAC: Adding query filter params (interface=public, service_id=c2935a12468840f3a06c3e2b3b3ce64f, region_id=RegionOne) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:36.778630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a154403-b3f5-4c2d-8237-7e480a26c52e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:36.780006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a154403-b3f5-4c2d-8237-7e480a26c52e None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:36.789257 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 644/1300] 10.222.0.115 () {56 vars in 1256 bytes} [Tue Aug 2 10:39:36 2022] GET /identity/v3/endpoints?service_id=c2935a12468840f3a06c3e2b3b3ce64f&interface=public®ion_id=RegionOne => generated 195 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:36.791374 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf6670-018f-4f21-af7c-c1ecb5b01713 None admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:36.874517 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:36.892460 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbf6670-018f-4f21-af7c-c1ecb5b01713 None admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:36.927848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3526b7-3dfb-4116-950d-1c9c80f236e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:36.928467 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3526b7-3dfb-4116-950d-1c9c80f236e9 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:36.928750 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3526b7-3dfb-4116-950d-1c9c80f236e9 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:36.929027 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3526b7-3dfb-4116-950d-1c9c80f236e9 None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:36.929706 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3526b7-3dfb-4116-950d-1c9c80f236e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:36.931134 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3526b7-3dfb-4116-950d-1c9c80f236e9 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:36.937798 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 657/1301] 10.222.0.115 () {56 vars in 1092 bytes} [Tue Aug 2 10:39:36 2022] GET /identity/v3/services => generated 2201 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:39:38.961329 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b41e1-cb0a-4743-8467-e04b3bca7d2e None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:38.961890 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b41e1-cb0a-4743-8467-e04b3bca7d2e None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:38.961890 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672b41e1-cb0a-4743-8467-e04b3bca7d2e None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:38.963283 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 645/1302] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:39:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:39:38.968550 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddd57c-ddff-4f1d-9315-b91de10660de None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:38.971472 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddd57c-ddff-4f1d-9315-b91de10660de None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:38.971472 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ddd57c-ddff-4f1d-9315-b91de10660de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:38.987406 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-c6ddd57c-ddff-4f1d-9315-b91de10660de None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:39:38.989289 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-c6ddd57c-ddff-4f1d-9315-b91de10660de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:39:39.064620 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 658/1303] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:39:38 2022] POST /identity/v3/auth/tokens => generated 2226 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 02 10:39:39.087509 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b41e1-cb0a-4743-8467-e04b3bca7d2e None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:39.184596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:39.191791 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672b41e1-cb0a-4743-8467-e04b3bca7d2e None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:39.228370 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf3695-1e1e-4872-accc-db31c52d007a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:39.228370 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf3695-1e1e-4872-accc-db31c52d007a None admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:39.228370 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf3695-1e1e-4872-accc-db31c52d007a None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:39.228370 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf3695-1e1e-4872-accc-db31c52d007a None admin] PATH_INFO: `/v3/services/placement` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:39.228814 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cf3695-1e1e-4872-accc-db31c52d007a None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:39.229726 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cf3695-1e1e-4872-accc-db31c52d007a None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:39.238658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application [None req-59cf3695-1e1e-4872-accc-db31c52d007a None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 10:39:39.238658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:39.238658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 02 10:39:39.238658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 02 10:39:39.238658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 02 10:39:39.238658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 02 10:39:39.238658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 02 10:39:39.238658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:39:39.238658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 10:39:39.238658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:39:39.238658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 02 10:39:39.238658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:39:39.238658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 02 10:39:39.238658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Aug 02 10:39:39.238658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 02 10:39:39.238658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Aug 02 10:39:39.238658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Aug 02 10:39:39.239946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 02 10:39:39.239946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 02 10:39:39.239946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Aug 02 10:39:39.239946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Aug 02 10:39:39.239946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 02 10:39:39.239946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 02 10:39:39.239946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 02 10:39:39.239946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 02 10:39:39.239946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 02 10:39:39.239946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 02 10:39:39.239946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 02 10:39:39.239946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 02 10:39:39.239946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 02 10:39:39.239946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 02 10:39:39.239946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 02 10:39:39.239946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.get_or_create( Aug 02 10:39:39.241007 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 02 10:39:39.241007 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application with Lock( Aug 02 10:39:39.241007 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 02 10:39:39.241007 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self._enter() Aug 02 10:39:39.241007 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 02 10:39:39.241007 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 02 10:39:39.241007 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 02 10:39:39.241007 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application return self.creator() Aug 02 10:39:39.241007 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 02 10:39:39.241007 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application created_value = creator( Aug 02 10:39:39.241007 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 02 10:39:39.241007 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 02 10:39:39.241007 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 02 10:39:39.241007 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: ERROR keystone.server.flask.application Aug 02 10:39:39.241007 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 646/1304] 10.222.0.115 () {56 vars in 1122 bytes} [Tue Aug 2 10:39:39 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 02 10:39:39.246007 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ddd57c-ddff-4f1d-9315-b91de10660de None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:39.326959 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:39.351267 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ddd57c-ddff-4f1d-9315-b91de10660de None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:39.385575 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e9ca09-ccb7-466f-b097-a2438bb846c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:39.386235 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9ca09-ccb7-466f-b097-a2438bb846c9 None admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:39.386519 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9ca09-ccb7-466f-b097-a2438bb846c9 None admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:39.386796 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9ca09-ccb7-466f-b097-a2438bb846c9 None admin] PATH_INFO: `/v3/services` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:39.387532 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e9ca09-ccb7-466f-b097-a2438bb846c9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:39:39.387838 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e9ca09-ccb7-466f-b097-a2438bb846c9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:39.389391 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e9ca09-ccb7-466f-b097-a2438bb846c9 None admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:39.395419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 659/1305] 10.222.0.115 () {56 vars in 1121 bytes} [Tue Aug 2 10:39:39 2022] GET /identity/v3/services?name=placement => generated 360 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:39:39.401967 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf3695-1e1e-4872-accc-db31c52d007a None admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:39:39.505104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf3695-1e1e-4872-accc-db31c52d007a None admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:39:39.538228 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07301d88-ecf2-460d-a21d-692698707173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:39:39.538827 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07301d88-ecf2-460d-a21d-692698707173 None admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:39:39.539129 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07301d88-ecf2-460d-a21d-692698707173 None admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:39:39.539400 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07301d88-ecf2-460d-a21d-692698707173 None admin] PATH_INFO: `/v3/endpoints` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:39:39.540104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07301d88-ecf2-460d-a21d-692698707173 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:39:39.541463 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07301d88-ecf2-460d-a21d-692698707173 None admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:39:39.620822 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 647/1306] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:39:39 2022] POST /identity/v3/endpoints => generated 333 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:39:49.392394 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=30 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:39:49.616203 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:46:15.017043 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae4b0b-9cab-4405-a83a-132e2c308451 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:15.018018 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae4b0b-9cab-4405-a83a-132e2c308451 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:15.018552 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae4b0b-9cab-4405-a83a-132e2c308451 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:15.021235 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 660/1307] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:46:15 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:46:15.026494 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef4258-6e8a-400d-b814-417838e4eaae None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:15.027438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef4258-6e8a-400d-b814-417838e4eaae None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:15.027970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ef4258-6e8a-400d-b814-417838e4eaae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:15.119420 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-b9ef4258-6e8a-400d-b814-417838e4eaae None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:46:15.121032 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-b9ef4258-6e8a-400d-b814-417838e4eaae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:15.259760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 648/1308] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:46:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:46:15.320720 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f449b1d-de43-4da9-9003-6856dc3c5f78 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:15.321018 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f449b1d-de43-4da9-9003-6856dc3c5f78 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:15.321627 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f449b1d-de43-4da9-9003-6856dc3c5f78 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:15.323452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 661/1309] 10.222.0.115 () {54 vars in 970 bytes} [Tue Aug 2 10:46:15 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:46:15.329601 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702a4c3-ab74-4080-a1a0-1436c521bfe3 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:15.330126 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702a4c3-ab74-4080-a1a0-1436c521bfe3 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:15.330571 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2702a4c3-ab74-4080-a1a0-1436c521bfe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:15.412741 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-2702a4c3-ab74-4080-a1a0-1436c521bfe3 None None] MFA Rules not processed for user `98dcb74152ec4375a586ea526e7f07c8`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:46:15.414040 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-2702a4c3-ab74-4080-a1a0-1436c521bfe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:15.443937 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:46:15.527185 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 649/1310] 10.222.0.115 () {58 vars in 1067 bytes} [Tue Aug 2 10:46:15 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:46:15.536526 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f449b1d-de43-4da9-9003-6856dc3c5f78 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:46:15.538687 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-6f449b1d-de43-4da9-9003-6856dc3c5f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:15.681536 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f449b1d-de43-4da9-9003-6856dc3c5f78 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:46:15.763513 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff21c498-fb42-444b-9c3b-18471045ca02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:46:15.764073 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21c498-fb42-444b-9c3b-18471045ca02 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:15.764363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21c498-fb42-444b-9c3b-18471045ca02 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:15.764634 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff21c498-fb42-444b-9c3b-18471045ca02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:15.766258 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-ff21c498-fb42-444b-9c3b-18471045ca02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:15.827571 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:46:15.885520 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff21c498-fb42-444b-9c3b-18471045ca02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:46:15.886960 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff21c498-fb42-444b-9c3b-18471045ca02 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:46:15.906762 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 662/1311] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:46:15 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:46:18.209162 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d9e60-73e4-42de-a13f-c184635c77f4 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:18.209735 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d9e60-73e4-42de-a13f-c184635c77f4 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:18.209735 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578d9e60-73e4-42de-a13f-c184635c77f4 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:18.211451 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 650/1312] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:46:18 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:46:18.216839 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc2819-70ba-4d9f-bf87-4b2b3a5998bf None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:18.217277 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc2819-70ba-4d9f-bf87-4b2b3a5998bf None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:18.217375 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc2819-70ba-4d9f-bf87-4b2b3a5998bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:18.312571 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-93bc2819-70ba-4d9f-bf87-4b2b3a5998bf None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:46:18.313814 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-93bc2819-70ba-4d9f-bf87-4b2b3a5998bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:18.426347 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 663/1313] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:46:18 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:46:18.465939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d9e60-73e4-42de-a13f-c184635c77f4 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:46:18.467561 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-578d9e60-73e4-42de-a13f-c184635c77f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:18.581939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578d9e60-73e4-42de-a13f-c184635c77f4 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:46:18.661125 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:46:18.663839 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706edfdb-7c02-4b6b-ad50-11f37362c245 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:46:18.664466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706edfdb-7c02-4b6b-ad50-11f37362c245 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:18.664744 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706edfdb-7c02-4b6b-ad50-11f37362c245 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:18.665016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706edfdb-7c02-4b6b-ad50-11f37362c245 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:18.666636 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-706edfdb-7c02-4b6b-ad50-11f37362c245 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:18.792345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706edfdb-7c02-4b6b-ad50-11f37362c245 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:46:18.794149 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706edfdb-7c02-4b6b-ad50-11f37362c245 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:46:18.814021 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 651/1314] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:46:18 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:46:21.348746 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721410f4-596a-43e8-b935-ff50d6cc6542 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:21.349348 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721410f4-596a-43e8-b935-ff50d6cc6542 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:21.349348 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721410f4-596a-43e8-b935-ff50d6cc6542 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:21.350652 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 664/1315] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:46:21 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:46:21.355807 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbbeb1-bcaf-4849-b693-26e0357448b1 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:21.356223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbbeb1-bcaf-4849-b693-26e0357448b1 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:21.356360 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fbbeb1-bcaf-4849-b693-26e0357448b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:21.421853 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-38fbbeb1-bcaf-4849-b693-26e0357448b1 None None] MFA Rules not processed for user `6636e4878e5846e598b80dcba8afbeef`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:46:21.423134 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-38fbbeb1-bcaf-4849-b693-26e0357448b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:21.481853 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:46:21.526273 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 652/1316] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:46:21 2022] POST /identity/v3/auth/tokens => generated 3253 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:46:21.579338 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3e096-a651-47e0-bd9d-2eeabcdeb48a None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:21.579630 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3e096-a651-47e0-bd9d-2eeabcdeb48a None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:21.579911 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3e096-a651-47e0-bd9d-2eeabcdeb48a None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:21.581283 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 665/1317] 10.222.0.115 () {54 vars in 970 bytes} [Tue Aug 2 10:46:21 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:46:21.586985 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a4a30a-c622-4565-911b-4ecaef9bded6 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:21.587291 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a4a30a-c622-4565-911b-4ecaef9bded6 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:21.587565 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a4a30a-c622-4565-911b-4ecaef9bded6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:21.676158 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-29a4a30a-c622-4565-911b-4ecaef9bded6 None None] MFA Rules not processed for user `98dcb74152ec4375a586ea526e7f07c8`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:46:21.676353 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-29a4a30a-c622-4565-911b-4ecaef9bded6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:21.755251 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 653/1318] 10.222.0.115 () {58 vars in 1067 bytes} [Tue Aug 2 10:46:21 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:46:21.764514 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d3e096-a651-47e0-bd9d-2eeabcdeb48a None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:46:21.767387 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-65d3e096-a651-47e0-bd9d-2eeabcdeb48a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:21.779792 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:46:21.882042 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d3e096-a651-47e0-bd9d-2eeabcdeb48a None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:46:21.955527 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e500f6c3-3cea-4e4e-995f-f0f0450e0101 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:46:21.956996 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e500f6c3-3cea-4e4e-995f-f0f0450e0101 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:21.957118 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e500f6c3-3cea-4e4e-995f-f0f0450e0101 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:21.957389 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e500f6c3-3cea-4e4e-995f-f0f0450e0101 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:21.959013 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-e500f6c3-3cea-4e4e-995f-f0f0450e0101 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:22.022254 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:46:22.071145 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e500f6c3-3cea-4e4e-995f-f0f0450e0101 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:46:22.072575 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e500f6c3-3cea-4e4e-995f-f0f0450e0101 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:46:22.091488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 666/1319] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:46:21 2022] GET /identity/v3/auth/tokens => generated 3253 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:46:25.347384 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518bc1c2-a54a-4d5b-963a-ccf194da1456 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:25.347928 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518bc1c2-a54a-4d5b-963a-ccf194da1456 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:25.347928 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518bc1c2-a54a-4d5b-963a-ccf194da1456 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:25.349310 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 654/1320] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:46:25 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:46:25.354511 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587d0d5-2a92-429b-9642-b23b7637e7d8 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:25.354900 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587d0d5-2a92-429b-9642-b23b7637e7d8 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:25.355074 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f587d0d5-2a92-429b-9642-b23b7637e7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:25.421014 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-f587d0d5-2a92-429b-9642-b23b7637e7d8 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:46:25.422224 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-f587d0d5-2a92-429b-9642-b23b7637e7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:25.520533 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 667/1321] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:46:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:46:25.559862 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518bc1c2-a54a-4d5b-963a-ccf194da1456 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:46:25.561394 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-518bc1c2-a54a-4d5b-963a-ccf194da1456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:25.610601 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:46:25.664022 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518bc1c2-a54a-4d5b-963a-ccf194da1456 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:46:25.738602 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11b4c4f-f990-4451-8af8-fc684cbd6cc1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:46:25.739282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b4c4f-f990-4451-8af8-fc684cbd6cc1 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:25.739566 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b4c4f-f990-4451-8af8-fc684cbd6cc1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:25.739837 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11b4c4f-f990-4451-8af8-fc684cbd6cc1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:25.741484 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-a11b4c4f-f990-4451-8af8-fc684cbd6cc1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:25.837780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:46:25.851500 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11b4c4f-f990-4451-8af8-fc684cbd6cc1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:46:25.852947 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11b4c4f-f990-4451-8af8-fc684cbd6cc1 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:46:25.871698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 655/1322] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:46:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:46:28.952997 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98eaf5-d96d-46b3-b613-2d45923a51a6 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:28.953537 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98eaf5-d96d-46b3-b613-2d45923a51a6 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:28.953537 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e98eaf5-d96d-46b3-b613-2d45923a51a6 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:28.954911 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 668/1323] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:46:28 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:46:28.960603 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602dd5b5-a77e-4350-85ac-0fbd22cc24cd None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:28.961037 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602dd5b5-a77e-4350-85ac-0fbd22cc24cd None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:28.961166 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602dd5b5-a77e-4350-85ac-0fbd22cc24cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:29.028467 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-602dd5b5-a77e-4350-85ac-0fbd22cc24cd None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:46:29.029680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-602dd5b5-a77e-4350-85ac-0fbd22cc24cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:29.144624 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 656/1324] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:46:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:46:29.183335 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e98eaf5-d96d-46b3-b613-2d45923a51a6 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:46:29.184893 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-2e98eaf5-d96d-46b3-b613-2d45923a51a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:29.206248 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:46:29.298817 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e98eaf5-d96d-46b3-b613-2d45923a51a6 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:46:29.378576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8fa34-be72-4ba3-9b89-ff2b971620e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:46:29.379223 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8fa34-be72-4ba3-9b89-ff2b971620e8 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:29.379508 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8fa34-be72-4ba3-9b89-ff2b971620e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:29.379783 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8fa34-be72-4ba3-9b89-ff2b971620e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:29.381434 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-2bb8fa34-be72-4ba3-9b89-ff2b971620e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:29.458233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:46:29.496606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb8fa34-be72-4ba3-9b89-ff2b971620e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:46:29.497944 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb8fa34-be72-4ba3-9b89-ff2b971620e8 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:46:29.515619 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 669/1325] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:46:29 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:46:31.621634 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc22a04-30f0-4432-bd8f-86ff64a2ddcb None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:31.622192 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc22a04-30f0-4432-bd8f-86ff64a2ddcb None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:31.622192 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc22a04-30f0-4432-bd8f-86ff64a2ddcb None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:31.623510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 657/1326] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:46:31 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:46:31.629599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1a571-d725-4475-95aa-c0450671ca15 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:31.630042 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1a571-d725-4475-95aa-c0450671ca15 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:31.630151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab1a571-d725-4475-95aa-c0450671ca15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:31.698147 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-6ab1a571-d725-4475-95aa-c0450671ca15 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:46:31.699397 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-6ab1a571-d725-4475-95aa-c0450671ca15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:31.797940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 670/1327] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:46:31 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:46:31.835425 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc22a04-30f0-4432-bd8f-86ff64a2ddcb None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:46:31.837149 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-5bc22a04-30f0-4432-bd8f-86ff64a2ddcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:31.898649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:46:31.943636 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc22a04-30f0-4432-bd8f-86ff64a2ddcb None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:46:32.022908 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05db1bec-7fec-4786-8141-7af4eb5dbc3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:46:32.023554 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db1bec-7fec-4786-8141-7af4eb5dbc3c service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:32.023834 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db1bec-7fec-4786-8141-7af4eb5dbc3c service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:32.024107 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05db1bec-7fec-4786-8141-7af4eb5dbc3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:32.025740 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-05db1bec-7fec-4786-8141-7af4eb5dbc3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:32.134501 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:46:32.137273 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db1bec-7fec-4786-8141-7af4eb5dbc3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:46:32.138722 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05db1bec-7fec-4786-8141-7af4eb5dbc3c service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:46:32.163924 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 658/1328] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:46:31 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:46:34.254797 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711e0c0-328e-43a0-9e65-ce1cc8e4e7fc None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:34.255421 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711e0c0-328e-43a0-9e65-ce1cc8e4e7fc None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:34.255421 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d711e0c0-328e-43a0-9e65-ce1cc8e4e7fc None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:34.256751 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 671/1329] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:46:34 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:46:34.262002 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66412c-c50b-4f62-8b55-75bc5ac5e481 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:34.262458 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66412c-c50b-4f62-8b55-75bc5ac5e481 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:34.262633 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66412c-c50b-4f62-8b55-75bc5ac5e481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:34.333796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-aa66412c-c50b-4f62-8b55-75bc5ac5e481 None None] MFA Rules not processed for user `6636e4878e5846e598b80dcba8afbeef`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:46:34.335038 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-aa66412c-c50b-4f62-8b55-75bc5ac5e481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:34.438405 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 659/1330] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:46:34 2022] POST /identity/v3/auth/tokens => generated 3253 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:46:34.476496 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711e0c0-328e-43a0-9e65-ce1cc8e4e7fc None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:46:34.478091 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-d711e0c0-328e-43a0-9e65-ce1cc8e4e7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:34.511769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:46:34.585126 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d711e0c0-328e-43a0-9e65-ce1cc8e4e7fc None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:46:34.660814 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14301b65-ab62-486b-bc94-9f823eb65dd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:46:34.661436 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14301b65-ab62-486b-bc94-9f823eb65dd5 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:34.661718 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14301b65-ab62-486b-bc94-9f823eb65dd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:34.661991 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14301b65-ab62-486b-bc94-9f823eb65dd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:34.663650 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-14301b65-ab62-486b-bc94-9f823eb65dd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:34.744385 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:46:34.776683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14301b65-ab62-486b-bc94-9f823eb65dd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:46:34.778138 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14301b65-ab62-486b-bc94-9f823eb65dd5 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:46:34.801927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 672/1331] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:46:34 2022] GET /identity/v3/auth/tokens => generated 3253 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:46:37.257219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ee2bd-b0a5-445e-af96-5794cad7b930 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:37.257726 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ee2bd-b0a5-445e-af96-5794cad7b930 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:37.257726 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3ee2bd-b0a5-445e-af96-5794cad7b930 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:37.259063 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 660/1332] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:46:37 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:46:37.264281 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da637a-6350-4228-8360-ec66dc78897a None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:37.264709 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da637a-6350-4228-8360-ec66dc78897a None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:37.264844 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62da637a-6350-4228-8360-ec66dc78897a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:37.332884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-62da637a-6350-4228-8360-ec66dc78897a None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:46:37.334095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-62da637a-6350-4228-8360-ec66dc78897a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:37.430246 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 673/1333] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:46:37 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:46:37.466602 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ee2bd-b0a5-445e-af96-5794cad7b930 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:46:37.468165 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-ce3ee2bd-b0a5-445e-af96-5794cad7b930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:37.537452 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:46:37.572010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3ee2bd-b0a5-445e-af96-5794cad7b930 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:46:37.644226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fd7358-0ba0-4ef6-ba26-2cd78431bd05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:46:37.644834 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd7358-0ba0-4ef6-ba26-2cd78431bd05 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:37.645113 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd7358-0ba0-4ef6-ba26-2cd78431bd05 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:37.645386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fd7358-0ba0-4ef6-ba26-2cd78431bd05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:37.647037 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-54fd7358-0ba0-4ef6-ba26-2cd78431bd05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:37.757359 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd7358-0ba0-4ef6-ba26-2cd78431bd05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:46:37.758798 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fd7358-0ba0-4ef6-ba26-2cd78431bd05 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:46:37.768919 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:46:37.776895 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 661/1334] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:46:37 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:46:39.843604 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cfd22d-f1ad-4b66-bbcf-595eae9b77ce None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:39.844152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cfd22d-f1ad-4b66-bbcf-595eae9b77ce None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:39.844152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cfd22d-f1ad-4b66-bbcf-595eae9b77ce None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:39.846132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 674/1335] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:46:39 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:46:39.851397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846ecb0-6a02-4163-9979-3c1da09ab392 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:39.851829 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846ecb0-6a02-4163-9979-3c1da09ab392 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:39.851959 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846ecb0-6a02-4163-9979-3c1da09ab392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:39.918361 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-6846ecb0-6a02-4163-9979-3c1da09ab392 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:46:39.919603 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-6846ecb0-6a02-4163-9979-3c1da09ab392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:40.014652 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 662/1336] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:46:39 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:46:40.056924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cfd22d-f1ad-4b66-bbcf-595eae9b77ce None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:46:40.058754 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-f3cfd22d-f1ad-4b66-bbcf-595eae9b77ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:40.099153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:46:40.161568 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cfd22d-f1ad-4b66-bbcf-595eae9b77ce None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:46:40.235453 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345954a1-840c-4145-b511-e411bd2fef4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:46:40.236734 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345954a1-840c-4145-b511-e411bd2fef4d service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:40.237084 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345954a1-840c-4145-b511-e411bd2fef4d service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:40.237363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345954a1-840c-4145-b511-e411bd2fef4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:40.239099 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-345954a1-840c-4145-b511-e411bd2fef4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:40.321776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:46:40.348749 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345954a1-840c-4145-b511-e411bd2fef4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:46:40.350241 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345954a1-840c-4145-b511-e411bd2fef4d service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:46:40.369612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 675/1337] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:46:40 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:46:43.136491 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677de6b-b017-47be-b3ec-86d168dee917 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:43.137042 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677de6b-b017-47be-b3ec-86d168dee917 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:43.137042 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677de6b-b017-47be-b3ec-86d168dee917 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:43.138408 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 663/1338] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:46:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:46:43.144597 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b61c0-d137-4e95-b949-d6d6735e192f None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:43.145264 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b61c0-d137-4e95-b949-d6d6735e192f None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:43.145513 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816b61c0-d137-4e95-b949-d6d6735e192f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:43.212227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-816b61c0-d137-4e95-b949-d6d6735e192f None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:46:43.213440 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-816b61c0-d137-4e95-b949-d6d6735e192f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:43.309229 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 676/1339] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:46:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:46:43.346168 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7677de6b-b017-47be-b3ec-86d168dee917 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:46:43.347717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-7677de6b-b017-47be-b3ec-86d168dee917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:43.426666 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:46:43.458023 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7677de6b-b017-47be-b3ec-86d168dee917 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:46:43.536600 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e7d624-0712-4e24-ad59-5004ba5e59e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:46:43.537221 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7d624-0712-4e24-ad59-5004ba5e59e5 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:43.537498 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7d624-0712-4e24-ad59-5004ba5e59e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:43.537772 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e7d624-0712-4e24-ad59-5004ba5e59e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:43.540133 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-85e7d624-0712-4e24-ad59-5004ba5e59e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:43.648359 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e7d624-0712-4e24-ad59-5004ba5e59e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:46:43.649786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e7d624-0712-4e24-ad59-5004ba5e59e5 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:46:43.667854 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 664/1340] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:46:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:46:47.984333 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a6f6ec-ab62-4ce0-a839-34e8af57ccc6 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:47.984878 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a6f6ec-ab62-4ce0-a839-34e8af57ccc6 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:47.984878 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a6f6ec-ab62-4ce0-a839-34e8af57ccc6 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:47.986847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 677/1341] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:46:47 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:46:47.992322 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01276bde-efd6-4a58-a4a0-eadabdce905e None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:47.992832 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01276bde-efd6-4a58-a4a0-eadabdce905e None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:47.992832 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01276bde-efd6-4a58-a4a0-eadabdce905e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:47.998486 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:46:48.065762 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-01276bde-efd6-4a58-a4a0-eadabdce905e None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:46:48.067006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-01276bde-efd6-4a58-a4a0-eadabdce905e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:48.168559 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 665/1342] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:46:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:46:48.208223 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a6f6ec-ab62-4ce0-a839-34e8af57ccc6 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:46:48.209818 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-77a6f6ec-ab62-4ce0-a839-34e8af57ccc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:48.258933 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:46:48.318092 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a6f6ec-ab62-4ce0-a839-34e8af57ccc6 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:46:48.395371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866960e5-cc28-46ad-b14b-42a94fae4347 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:46:48.395992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866960e5-cc28-46ad-b14b-42a94fae4347 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:48.396275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866960e5-cc28-46ad-b14b-42a94fae4347 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:48.396551 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866960e5-cc28-46ad-b14b-42a94fae4347 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:48.398263 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-866960e5-cc28-46ad-b14b-42a94fae4347 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:48.499516 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:46:48.519574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866960e5-cc28-46ad-b14b-42a94fae4347 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:46:48.519574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866960e5-cc28-46ad-b14b-42a94fae4347 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:46:48.537755 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 678/1343] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:46:48 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:46:51.579085 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21370e-0deb-434c-adf3-474c9a542fc2 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:51.580430 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21370e-0deb-434c-adf3-474c9a542fc2 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:51.580522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae21370e-0deb-434c-adf3-474c9a542fc2 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:51.581937 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 666/1344] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:46:51 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:46:51.587448 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda1b05-60d2-46e2-8a42-0ef9b4821b02 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:51.587980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda1b05-60d2-46e2-8a42-0ef9b4821b02 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:51.588140 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda1b05-60d2-46e2-8a42-0ef9b4821b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:51.656401 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-4bda1b05-60d2-46e2-8a42-0ef9b4821b02 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:46:51.657655 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-4bda1b05-60d2-46e2-8a42-0ef9b4821b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:51.763722 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 679/1345] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:46:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:46:51.803193 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21370e-0deb-434c-adf3-474c9a542fc2 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:46:51.804711 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-ae21370e-0deb-434c-adf3-474c9a542fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:51.894426 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:46:51.912400 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae21370e-0deb-434c-adf3-474c9a542fc2 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:46:51.993183 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a70f06-c81c-4084-804f-e6791651c513 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:46:51.993854 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a70f06-c81c-4084-804f-e6791651c513 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:51.994124 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a70f06-c81c-4084-804f-e6791651c513 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:51.994394 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a70f06-c81c-4084-804f-e6791651c513 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:51.996080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-d0a70f06-c81c-4084-804f-e6791651c513 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:52.121281 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a70f06-c81c-4084-804f-e6791651c513 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:46:52.122740 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a70f06-c81c-4084-804f-e6791651c513 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:46:52.142841 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 667/1346] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:46:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:46:56.607803 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e070aa-b26f-4dd1-83c8-4a76dd121d8a None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:56.608332 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e070aa-b26f-4dd1-83c8-4a76dd121d8a None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:56.608332 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e070aa-b26f-4dd1-83c8-4a76dd121d8a None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:56.609738 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 680/1347] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:46:56 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:46:56.616224 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a77d6a-7458-4dc0-a8bd-209f54f0fc45 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:56.616810 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a77d6a-7458-4dc0-a8bd-209f54f0fc45 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:56.616909 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a77d6a-7458-4dc0-a8bd-209f54f0fc45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:56.634344 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:46:56.691459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-63a77d6a-7458-4dc0-a8bd-209f54f0fc45 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:46:56.692756 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-63a77d6a-7458-4dc0-a8bd-209f54f0fc45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:56.813405 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 668/1348] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:46:56 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:46:56.854071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e070aa-b26f-4dd1-83c8-4a76dd121d8a None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:46:56.855687 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-04e070aa-b26f-4dd1-83c8-4a76dd121d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:56.927466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:46:56.987363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e070aa-b26f-4dd1-83c8-4a76dd121d8a None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:46:57.073746 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f760ee-1f69-49cc-80c5-1911402bad61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:46:57.074371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f760ee-1f69-49cc-80c5-1911402bad61 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:46:57.074651 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f760ee-1f69-49cc-80c5-1911402bad61 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:46:57.075139 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f760ee-1f69-49cc-80c5-1911402bad61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:46:57.076651 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-a0f760ee-1f69-49cc-80c5-1911402bad61 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:46:57.200549 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:46:57.204284 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f760ee-1f69-49cc-80c5-1911402bad61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:46:57.205768 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f760ee-1f69-49cc-80c5-1911402bad61 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:46:57.225672 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 681/1349] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:46:56 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:47:02.253487 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3a0ea-d928-45dd-9c38-3c4fb6369fad None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:02.254046 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3a0ea-d928-45dd-9c38-3c4fb6369fad None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:02.254288 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d3a0ea-d928-45dd-9c38-3c4fb6369fad None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:02.256033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 669/1350] 10.222.0.115 () {54 vars in 926 bytes} [Tue Aug 2 10:47:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:47:02.261211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd738a-88ee-4576-8206-2b4bfcdd1b39 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:02.261677 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd738a-88ee-4576-8206-2b4bfcdd1b39 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:02.261775 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd738a-88ee-4576-8206-2b4bfcdd1b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:02.415354 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-39dd738a-88ee-4576-8206-2b4bfcdd1b39 None None] MFA Rules not processed for user `cbc4b962e2a742b08beef7187cc527e5`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:47:02.416579 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-39dd738a-88ee-4576-8206-2b4bfcdd1b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:02.553820 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 682/1351] 10.222.0.115 () {58 vars in 1023 bytes} [Tue Aug 2 10:47:02 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:47:06.614225 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111eed5-4249-431e-b0b4-0cbd9f62ce43 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:06.614788 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111eed5-4249-431e-b0b4-0cbd9f62ce43 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:06.614788 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111eed5-4249-431e-b0b4-0cbd9f62ce43 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:06.616480 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 670/1352] 10.222.0.115 () {54 vars in 926 bytes} [Tue Aug 2 10:47:06 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:47:06.621671 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec9509-8d03-42af-8c23-3c5f6ba0bcec None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:06.622128 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec9509-8d03-42af-8c23-3c5f6ba0bcec None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:06.622217 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec9509-8d03-42af-8c23-3c5f6ba0bcec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:06.680834 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:06.705978 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-cdec9509-8d03-42af-8c23-3c5f6ba0bcec None None] MFA Rules not processed for user `cbc4b962e2a742b08beef7187cc527e5`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:47:06.707252 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-cdec9509-8d03-42af-8c23-3c5f6ba0bcec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:06.811141 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=59 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:06.825598 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 683/1353] 10.222.0.115 () {58 vars in 1023 bytes} [Tue Aug 2 10:47:06 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:47:08.273002 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663a113-df1a-4ae9-94c4-bfe973d5bb45 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:08.273552 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663a113-df1a-4ae9-94c4-bfe973d5bb45 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:08.273552 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9663a113-df1a-4ae9-94c4-bfe973d5bb45 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:08.274898 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 671/1354] 10.222.0.115 () {54 vars in 926 bytes} [Tue Aug 2 10:47:08 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:47:08.279932 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0dacb-7b84-4494-9f45-47fcd3bae792 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:08.280352 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0dacb-7b84-4494-9f45-47fcd3bae792 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:08.280495 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b0dacb-7b84-4494-9f45-47fcd3bae792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:08.360057 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-c2b0dacb-7b84-4494-9f45-47fcd3bae792 None None] MFA Rules not processed for user `cbc4b962e2a742b08beef7187cc527e5`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:47:08.361248 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-c2b0dacb-7b84-4494-9f45-47fcd3bae792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:08.402896 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:08.474994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 684/1355] 10.222.0.115 () {58 vars in 1023 bytes} [Tue Aug 2 10:47:08 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:47:11.908926 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a58c9-082e-4c41-9c35-864ab769cc1d None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:11.909814 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a58c9-082e-4c41-9c35-864ab769cc1d None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:11.910267 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824a58c9-082e-4c41-9c35-864ab769cc1d None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:11.913497 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 672/1356] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:47:11 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:47:11.917898 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b762c4e-c69c-4524-a576-7979c6b19d73 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:11.918937 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b762c4e-c69c-4524-a576-7979c6b19d73 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:11.919471 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b762c4e-c69c-4524-a576-7979c6b19d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:12.009157 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-7b762c4e-c69c-4524-a576-7979c6b19d73 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:47:12.010642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-7b762c4e-c69c-4524-a576-7979c6b19d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:12.122926 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 685/1357] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:47:11 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:47:12.177798 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac333d0-9453-415b-a2c6-87f78f10815d None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:12.178565 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac333d0-9453-415b-a2c6-87f78f10815d None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:12.179234 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac333d0-9453-415b-a2c6-87f78f10815d None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:12.180934 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 673/1358] 10.222.0.115 () {54 vars in 967 bytes} [Tue Aug 2 10:47:12 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:47:12.190658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741c84e-ccd4-472c-a7cc-635f9320e0fb None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:12.190960 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741c84e-ccd4-472c-a7cc-635f9320e0fb None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:12.191261 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9741c84e-ccd4-472c-a7cc-635f9320e0fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:12.215646 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:12.287557 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-9741c84e-ccd4-472c-a7cc-635f9320e0fb None None] MFA Rules not processed for user `d0478bd540f446ec810e2091c46ac49c`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:47:12.289094 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-9741c84e-ccd4-472c-a7cc-635f9320e0fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:12.428548 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 686/1359] 10.222.0.115 () {58 vars in 1064 bytes} [Tue Aug 2 10:47:12 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:47:12.438357 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac333d0-9453-415b-a2c6-87f78f10815d None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:47:12.439878 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-eac333d0-9453-415b-a2c6-87f78f10815d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:12.607752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac333d0-9453-415b-a2c6-87f78f10815d None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:47:12.686103 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:12.729376 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cc5d51-0fb7-4f49-bed4-8238c4895a5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:47:12.730173 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc5d51-0fb7-4f49-bed4-8238c4895a5f service nova] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:12.730518 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc5d51-0fb7-4f49-bed4-8238c4895a5f service nova] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:12.730889 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cc5d51-0fb7-4f49-bed4-8238c4895a5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:12.732640 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f9cc5d51-0fb7-4f49-bed4-8238c4895a5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:12.833753 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cc5d51-0fb7-4f49-bed4-8238c4895a5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:47:12.835507 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cc5d51-0fb7-4f49-bed4-8238c4895a5f service nova] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:47:12.854931 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 674/1360] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:47:12 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:47:15.106257 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5e01d-d159-4eb3-99d3-c8a62109aa25 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:15.106828 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5e01d-d159-4eb3-99d3-c8a62109aa25 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:15.106828 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed5e01d-d159-4eb3-99d3-c8a62109aa25 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:15.108233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 687/1361] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:47:15 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:47:15.114202 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58ccdd-01e3-4272-af61-cca69226098a None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:15.114772 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58ccdd-01e3-4272-af61-cca69226098a None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:15.114772 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58ccdd-01e3-4272-af61-cca69226098a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:15.185217 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-9c58ccdd-01e3-4272-af61-cca69226098a None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:47:15.186419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-9c58ccdd-01e3-4272-af61-cca69226098a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:15.230518 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:15.298382 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 675/1362] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:47:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:47:15.344364 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5e01d-d159-4eb3-99d3-c8a62109aa25 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:47:15.346063 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-2ed5e01d-d159-4eb3-99d3-c8a62109aa25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:15.437582 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:15.538030 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed5e01d-d159-4eb3-99d3-c8a62109aa25 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:47:15.707614 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4e0f1f-448b-4397-b3a3-b97cfe9ea2cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:47:15.711519 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e0f1f-448b-4397-b3a3-b97cfe9ea2cb service nova] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:15.711519 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e0f1f-448b-4397-b3a3-b97cfe9ea2cb service nova] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:15.711519 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4e0f1f-448b-4397-b3a3-b97cfe9ea2cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:15.711519 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-2e4e0f1f-448b-4397-b3a3-b97cfe9ea2cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:15.734654 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:15.805590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4e0f1f-448b-4397-b3a3-b97cfe9ea2cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:47:15.807643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4e0f1f-448b-4397-b3a3-b97cfe9ea2cb service nova] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:47:15.829922 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 688/1363] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:47:15 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:47:18.311612 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd5872-f58d-4239-b0ac-522ddf2392cd None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:18.312763 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd5872-f58d-4239-b0ac-522ddf2392cd None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:18.312763 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bd5872-f58d-4239-b0ac-522ddf2392cd None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:18.314093 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 676/1364] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:47:18 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:47:18.321159 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90d499-4f07-4765-ab24-299135c8cc07 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:18.321654 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90d499-4f07-4765-ab24-299135c8cc07 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:18.321837 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d90d499-4f07-4765-ab24-299135c8cc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:18.394813 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-0d90d499-4f07-4765-ab24-299135c8cc07 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:47:18.396101 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-0d90d499-4f07-4765-ab24-299135c8cc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:18.507445 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:18.516086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 689/1365] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:47:18 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:47:18.560806 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9776c6a-c83f-472a-bff4-876a06a93085 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:18.561349 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9776c6a-c83f-472a-bff4-876a06a93085 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:18.561832 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9776c6a-c83f-472a-bff4-876a06a93085 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:18.564431 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 677/1366] 10.222.0.115 () {54 vars in 967 bytes} [Tue Aug 2 10:47:18 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:47:18.569987 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd654d-5d63-4426-ab5f-7f0d29b203a2 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:18.570356 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd654d-5d63-4426-ab5f-7f0d29b203a2 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:18.570711 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbd654d-5d63-4426-ab5f-7f0d29b203a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:18.655293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-cfbd654d-5d63-4426-ab5f-7f0d29b203a2 None None] MFA Rules not processed for user `d0478bd540f446ec810e2091c46ac49c`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:47:18.657798 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-cfbd654d-5d63-4426-ab5f-7f0d29b203a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:18.799282 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 690/1367] 10.222.0.115 () {58 vars in 1064 bytes} [Tue Aug 2 10:47:18 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:47:18.808304 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9776c6a-c83f-472a-bff4-876a06a93085 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:47:18.810072 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-d9776c6a-c83f-472a-bff4-876a06a93085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:18.970277 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9776c6a-c83f-472a-bff4-876a06a93085 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:47:18.992797 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:19.095738 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a1b3c-0f4c-4a5f-bee4-9383d478a86b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:47:19.096363 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a1b3c-0f4c-4a5f-bee4-9383d478a86b service nova] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:19.096641 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a1b3c-0f4c-4a5f-bee4-9383d478a86b service nova] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:19.096913 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a1b3c-0f4c-4a5f-bee4-9383d478a86b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:19.098542 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-de2a1b3c-0f4c-4a5f-bee4-9383d478a86b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:19.195077 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2a1b3c-0f4c-4a5f-bee4-9383d478a86b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:47:19.196704 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2a1b3c-0f4c-4a5f-bee4-9383d478a86b service nova] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:47:19.216518 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 678/1368] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:47:18 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:47:20.140206 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcb6fb-2e57-43be-aa15-8bbe7cfa7c49 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:20.141422 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcb6fb-2e57-43be-aa15-8bbe7cfa7c49 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:20.141629 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcb6fb-2e57-43be-aa15-8bbe7cfa7c49 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:20.143233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 691/1369] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:47:20 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:47:20.148371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352e49e-fb9c-4660-92f9-9f7ccb7ce02d None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:20.148840 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352e49e-fb9c-4660-92f9-9f7ccb7ce02d None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:20.148938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352e49e-fb9c-4660-92f9-9f7ccb7ce02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:20.182725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:20.219457 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-2352e49e-fb9c-4660-92f9-9f7ccb7ce02d None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:47:20.220691 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-2352e49e-fb9c-4660-92f9-9f7ccb7ce02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:20.325350 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 679/1370] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:47:20 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:47:20.675407 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1fbff-ee90-46d2-a29e-33acb051b19f None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:20.675407 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1fbff-ee90-46d2-a29e-33acb051b19f None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:20.675407 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1fbff-ee90-46d2-a29e-33acb051b19f None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:20.675943 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 692/1371] 10.222.0.115 () {54 vars in 969 bytes} [Tue Aug 2 10:47:20 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:47:20.680629 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7aa74-85e8-43b8-8987-6e2bee911df9 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:20.680925 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7aa74-85e8-43b8-8987-6e2bee911df9 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:20.681456 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a7aa74-85e8-43b8-8987-6e2bee911df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:20.767026 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-75a7aa74-85e8-43b8-8987-6e2bee911df9 None None] MFA Rules not processed for user `cc9ab3c9075b4a8385ef634dc0b1625e`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:47:20.768413 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-75a7aa74-85e8-43b8-8987-6e2bee911df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:20.818965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:20.849288 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 680/1372] 10.222.0.115 () {58 vars in 1066 bytes} [Tue Aug 2 10:47:20 2022] POST /identity/v3/auth/tokens => generated 3130 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:47:20.856819 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1fbff-ee90-46d2-a29e-33acb051b19f None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:47:20.858341 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-aad1fbff-ee90-46d2-a29e-33acb051b19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:20.914092 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:20.973103 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1fbff-ee90-46d2-a29e-33acb051b19f None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:47:21.053105 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98523e1c-3e53-403d-97c5-6cb934ab6da8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:47:21.053682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98523e1c-3e53-403d-97c5-6cb934ab6da8 service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:21.053916 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98523e1c-3e53-403d-97c5-6cb934ab6da8 service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:21.054191 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98523e1c-3e53-403d-97c5-6cb934ab6da8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:21.055847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-98523e1c-3e53-403d-97c5-6cb934ab6da8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:21.160685 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:21.176086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98523e1c-3e53-403d-97c5-6cb934ab6da8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:47:21.177719 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98523e1c-3e53-403d-97c5-6cb934ab6da8 service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:47:21.200393 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 693/1373] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:47:20 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:47:21.488877 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80ca7c-acb0-4d5c-98f7-1711b948c917 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:21.489309 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80ca7c-acb0-4d5c-98f7-1711b948c917 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:21.489477 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80ca7c-acb0-4d5c-98f7-1711b948c917 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:21.491378 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 681/1374] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:47:21 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:47:21.497362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a84d9-7547-44b9-bbf2-207c78bd4c9c None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:21.497924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a84d9-7547-44b9-bbf2-207c78bd4c9c None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:21.498395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335a84d9-7547-44b9-bbf2-207c78bd4c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:21.600565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-335a84d9-7547-44b9-bbf2-207c78bd4c9c None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:47:21.602575 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-335a84d9-7547-44b9-bbf2-207c78bd4c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:21.718743 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 694/1375] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:47:21 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:47:21.764408 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80ca7c-acb0-4d5c-98f7-1711b948c917 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:47:21.766146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-0b80ca7c-acb0-4d5c-98f7-1711b948c917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:21.915168 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80ca7c-acb0-4d5c-98f7-1711b948c917 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:47:21.973511 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:22.036771 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02050792-feca-4d2c-907d-273fb5c194a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:47:22.037389 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02050792-feca-4d2c-907d-273fb5c194a2 service nova] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:22.037675 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02050792-feca-4d2c-907d-273fb5c194a2 service nova] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:22.037943 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02050792-feca-4d2c-907d-273fb5c194a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:22.039590 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-02050792-feca-4d2c-907d-273fb5c194a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:22.130209 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02050792-feca-4d2c-907d-273fb5c194a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:47:22.131759 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02050792-feca-4d2c-907d-273fb5c194a2 service nova] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:47:22.149619 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 682/1376] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:47:21 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:47:24.336354 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb247ff-cf30-4790-a0ce-9c0c418024a4 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:24.336868 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb247ff-cf30-4790-a0ce-9c0c418024a4 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:24.336954 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb247ff-cf30-4790-a0ce-9c0c418024a4 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:24.340306 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 695/1377] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:47:24 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:47:24.344552 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c59719-08e6-4bda-8ef3-c558a4d9ee05 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:24.345045 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c59719-08e6-4bda-8ef3-c558a4d9ee05 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:24.345155 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c59719-08e6-4bda-8ef3-c558a4d9ee05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:24.414803 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-08c59719-08e6-4bda-8ef3-c558a4d9ee05 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:47:24.415950 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-08c59719-08e6-4bda-8ef3-c558a4d9ee05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:24.422787 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:24.516428 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 683/1378] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:47:24 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:47:24.561607 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb247ff-cf30-4790-a0ce-9c0c418024a4 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:47:24.563142 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-aeb247ff-cf30-4790-a0ce-9c0c418024a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:24.632575 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:24.737946 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb247ff-cf30-4790-a0ce-9c0c418024a4 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:47:24.885776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baec08e2-eb59-47b4-bf73-fb17a7caccab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:47:24.886488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec08e2-eb59-47b4-bf73-fb17a7caccab service nova] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:24.887038 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec08e2-eb59-47b4-bf73-fb17a7caccab service nova] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:24.887454 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baec08e2-eb59-47b4-bf73-fb17a7caccab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:24.889527 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-baec08e2-eb59-47b4-bf73-fb17a7caccab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:24.912816 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:24.997835 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baec08e2-eb59-47b4-bf73-fb17a7caccab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:47:24.999695 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baec08e2-eb59-47b4-bf73-fb17a7caccab service nova] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:47:25.019008 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 696/1379] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:47:24 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:47:27.220858 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc3b29-47ca-41b9-a346-a5c727089495 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:27.221288 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc3b29-47ca-41b9-a346-a5c727089495 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:27.221569 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfc3b29-47ca-41b9-a346-a5c727089495 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:27.223393 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 684/1380] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:47:27 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:47:27.228016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bd6ce-5a00-4d48-a59f-863826d4f347 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:27.228435 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bd6ce-5a00-4d48-a59f-863826d4f347 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:27.228633 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bd6ce-5a00-4d48-a59f-863826d4f347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:27.303028 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-352bd6ce-5a00-4d48-a59f-863826d4f347 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:47:27.304278 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-352bd6ce-5a00-4d48-a59f-863826d4f347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:27.412721 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:27.419093 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 697/1381] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:47:27 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:47:27.466802 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc3b29-47ca-41b9-a346-a5c727089495 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:47:27.469178 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-ddfc3b29-47ca-41b9-a346-a5c727089495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:27.614650 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:27.625729 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfc3b29-47ca-41b9-a346-a5c727089495 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:47:27.766068 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a447ef4c-007b-4f90-90ef-a1db9f1a4a93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:47:27.766426 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447ef4c-007b-4f90-90ef-a1db9f1a4a93 service nova] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:27.766694 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447ef4c-007b-4f90-90ef-a1db9f1a4a93 service nova] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:27.766979 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a447ef4c-007b-4f90-90ef-a1db9f1a4a93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:27.769641 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-a447ef4c-007b-4f90-90ef-a1db9f1a4a93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:27.867517 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a447ef4c-007b-4f90-90ef-a1db9f1a4a93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:47:27.869226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a447ef4c-007b-4f90-90ef-a1db9f1a4a93 service nova] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:47:27.881090 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:27.889220 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 685/1382] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:47:27 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:47:30.078867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb32df-0a8f-43ee-8d8c-e78fb5ac09bd None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:30.079448 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb32df-0a8f-43ee-8d8c-e78fb5ac09bd None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:30.079448 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cb32df-0a8f-43ee-8d8c-e78fb5ac09bd None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:30.081024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 698/1383] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:47:30 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:47:30.086001 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a1d13-bc80-4f36-a52b-54de19aed63f None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:30.086435 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a1d13-bc80-4f36-a52b-54de19aed63f None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:30.086531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a1d13-bc80-4f36-a52b-54de19aed63f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:30.162267 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-420a1d13-bc80-4f36-a52b-54de19aed63f None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:47:30.163498 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-420a1d13-bc80-4f36-a52b-54de19aed63f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:30.277232 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 686/1384] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:47:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:47:30.320857 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb32df-0a8f-43ee-8d8c-e78fb5ac09bd None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:47:30.322400 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-e7cb32df-0a8f-43ee-8d8c-e78fb5ac09bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:30.481487 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cb32df-0a8f-43ee-8d8c-e78fb5ac09bd None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:47:30.572515 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:30.633216 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b688be62-03b6-4239-ad6a-a93d848814fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:47:30.634269 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688be62-03b6-4239-ad6a-a93d848814fc service nova] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:30.634925 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688be62-03b6-4239-ad6a-a93d848814fc service nova] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:30.635544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688be62-03b6-4239-ad6a-a93d848814fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:30.637835 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-b688be62-03b6-4239-ad6a-a93d848814fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:30.730792 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b688be62-03b6-4239-ad6a-a93d848814fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:47:30.732399 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b688be62-03b6-4239-ad6a-a93d848814fc service nova] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:47:30.751144 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 699/1385] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:47:30 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:47:32.787089 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01f510-cb9d-47eb-bfb8-4939aceb2163 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:32.787706 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01f510-cb9d-47eb-bfb8-4939aceb2163 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:32.787706 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a01f510-cb9d-47eb-bfb8-4939aceb2163 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:32.790690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 687/1386] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:47:32 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:47:32.794815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a139cb-07b6-4a87-a009-15b3e2e584a2 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:32.795456 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a139cb-07b6-4a87-a009-15b3e2e584a2 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:32.795456 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a139cb-07b6-4a87-a009-15b3e2e584a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:32.865661 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-b7a139cb-07b6-4a87-a009-15b3e2e584a2 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:47:32.866890 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-b7a139cb-07b6-4a87-a009-15b3e2e584a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:32.900446 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:32.967171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 700/1387] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:47:32 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:47:33.012217 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a01f510-cb9d-47eb-bfb8-4939aceb2163 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:47:33.013737 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-3a01f510-cb9d-47eb-bfb8-4939aceb2163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:33.097231 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:33.173327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a01f510-cb9d-47eb-bfb8-4939aceb2163 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:47:33.289946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7f02a1-b76b-4bb3-8d40-35cbd34abef5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:47:33.290568 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f02a1-b76b-4bb3-8d40-35cbd34abef5 service nova] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:33.290868 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f02a1-b76b-4bb3-8d40-35cbd34abef5 service nova] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:33.291155 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7f02a1-b76b-4bb3-8d40-35cbd34abef5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:33.292797 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-5f7f02a1-b76b-4bb3-8d40-35cbd34abef5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:33.341394 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:33.402362 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7f02a1-b76b-4bb3-8d40-35cbd34abef5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:47:33.404104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7f02a1-b76b-4bb3-8d40-35cbd34abef5 service nova] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:47:33.422820 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 688/1388] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:47:33 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:47:35.651765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27084c-5b5f-40e0-af06-e06dd833d50c None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:35.652388 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27084c-5b5f-40e0-af06-e06dd833d50c None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:35.652388 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f27084c-5b5f-40e0-af06-e06dd833d50c None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:35.653693 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 701/1389] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:47:35 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:47:35.659779 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab066d1-1e94-4bd5-830c-1441c9b1d01d None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:35.660296 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab066d1-1e94-4bd5-830c-1441c9b1d01d None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:35.660403 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab066d1-1e94-4bd5-830c-1441c9b1d01d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:35.737431 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-5ab066d1-1e94-4bd5-830c-1441c9b1d01d None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:47:35.738532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-5ab066d1-1e94-4bd5-830c-1441c9b1d01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:35.841124 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 689/1390] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:47:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:47:35.886221 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f27084c-5b5f-40e0-af06-e06dd833d50c None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:47:35.890208 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-5f27084c-5b5f-40e0-af06-e06dd833d50c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:36.162684 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f27084c-5b5f-40e0-af06-e06dd833d50c None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:47:36.179954 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:36.284218 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb188bf-b872-4186-94cf-9d8a854b0675 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:47:36.284839 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb188bf-b872-4186-94cf-9d8a854b0675 service nova] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:36.285117 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb188bf-b872-4186-94cf-9d8a854b0675 service nova] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:36.285395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb188bf-b872-4186-94cf-9d8a854b0675 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:36.287001 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-fcb188bf-b872-4186-94cf-9d8a854b0675 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:36.378128 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb188bf-b872-4186-94cf-9d8a854b0675 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:47:36.380847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb188bf-b872-4186-94cf-9d8a854b0675 service nova] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:47:36.398884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 702/1391] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:47:35 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 513 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:47:36.639005 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff1ed0-cc72-4bed-81c1-29d922283fac None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:36.639462 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff1ed0-cc72-4bed-81c1-29d922283fac None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:36.639849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ff1ed0-cc72-4bed-81c1-29d922283fac None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:36.641314 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 690/1392] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:47:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:47:36.647672 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d56ff-91b1-49ed-9223-370d53e38635 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:36.647960 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d56ff-91b1-49ed-9223-370d53e38635 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:36.648254 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0d56ff-91b1-49ed-9223-370d53e38635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:36.678075 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:36.726775 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-7e0d56ff-91b1-49ed-9223-370d53e38635 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:47:36.727953 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-7e0d56ff-91b1-49ed-9223-370d53e38635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:36.829211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 703/1393] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:47:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:47:36.887226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adff09e-0da0-4e03-8cd2-ccb924816982 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:36.887726 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adff09e-0da0-4e03-8cd2-ccb924816982 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:36.887828 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adff09e-0da0-4e03-8cd2-ccb924816982 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:36.889130 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 691/1394] 10.222.0.115 () {54 vars in 971 bytes} [Tue Aug 2 10:47:36 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:47:36.895378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18afed02-99ea-418c-a797-630f23c4ccf6 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:36.895668 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18afed02-99ea-418c-a797-630f23c4ccf6 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:36.895946 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18afed02-99ea-418c-a797-630f23c4ccf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:36.974030 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-18afed02-99ea-418c-a797-630f23c4ccf6 None None] MFA Rules not processed for user `e358fd7c1b3c43d7ac89fb3562678266`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:47:36.975256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-18afed02-99ea-418c-a797-630f23c4ccf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:37.017288 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:37.054344 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 704/1395] 10.222.0.115 () {58 vars in 1068 bytes} [Tue Aug 2 10:47:36 2022] POST /identity/v3/auth/tokens => generated 3130 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:47:37.062838 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adff09e-0da0-4e03-8cd2-ccb924816982 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:47:37.064448 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-0adff09e-0da0-4e03-8cd2-ccb924816982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:37.078663 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:37.172844 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adff09e-0da0-4e03-8cd2-ccb924816982 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:47:37.252010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce432f9a-2b5f-4adb-b11d-0612e14bc5a2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e358fd7c1b3c43d7ac89fb3562678266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:47:37.252627 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce432f9a-2b5f-4adb-b11d-0612e14bc5a2 service glance] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:37.253139 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce432f9a-2b5f-4adb-b11d-0612e14bc5a2 service glance] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:37.253422 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce432f9a-2b5f-4adb-b11d-0612e14bc5a2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:37.255282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-ce432f9a-2b5f-4adb-b11d-0612e14bc5a2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:37.325303 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:37.366629 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce432f9a-2b5f-4adb-b11d-0612e14bc5a2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:47:37.368114 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce432f9a-2b5f-4adb-b11d-0612e14bc5a2 service glance] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:47:37.386727 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 692/1396] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:47:37 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:47:38.306607 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a9d5a-c47f-4f72-b06d-e71d52040fbf None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:38.310598 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a9d5a-c47f-4f72-b06d-e71d52040fbf None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:38.310598 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2a9d5a-c47f-4f72-b06d-e71d52040fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:38.385357 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-3a2a9d5a-c47f-4f72-b06d-e71d52040fbf None None] MFA Rules not processed for user `2b03263937cb4ca7807a24da44d37b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:47:38.386627 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-3a2a9d5a-c47f-4f72-b06d-e71d52040fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:38.490414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 705/1397] 10.222.0.115 () {58 vars in 1014 bytes} [Tue Aug 2 10:47:38 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:47:38.541255 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514f006-6026-4c78-b1e9-5905201a94eb None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:38.541805 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514f006-6026-4c78-b1e9-5905201a94eb None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:38.541805 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5514f006-6026-4c78-b1e9-5905201a94eb None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:38.543458 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 693/1398] 10.222.0.115 () {54 vars in 948 bytes} [Tue Aug 2 10:47:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:47:38.550840 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c49a98-0f16-46fc-910e-b274745ea7fc None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:38.551244 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c49a98-0f16-46fc-910e-b274745ea7fc None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:38.551616 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c49a98-0f16-46fc-910e-b274745ea7fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:38.611940 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b993ff-6a01-4e04-831d-e23a16738c31 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:38.612226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b993ff-6a01-4e04-831d-e23a16738c31 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:38.612530 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b993ff-6a01-4e04-831d-e23a16738c31 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:38.613902 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 694/1399] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:47:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:47:38.619344 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf7831-7f22-405b-b3ca-d729f42e2fe0 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:38.619629 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf7831-7f22-405b-b3ca-d729f42e2fe0 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:38.619904 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bf7831-7f22-405b-b3ca-d729f42e2fe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:38.637626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:38.645897 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-02c49a98-0f16-46fc-910e-b274745ea7fc None None] MFA Rules not processed for user `0daa49b5a35f4a468bb8d0939e7ec20c`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:47:38.647183 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-02c49a98-0f16-46fc-910e-b274745ea7fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:38.695493 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-76bf7831-7f22-405b-b3ca-d729f42e2fe0 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:47:38.696781 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-76bf7831-7f22-405b-b3ca-d729f42e2fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:38.775333 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 706/1400] 10.222.0.115 () {58 vars in 1045 bytes} [Tue Aug 2 10:47:38 2022] POST /identity/v3/auth/tokens => generated 3314 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:47:38.785880 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c49a98-0f16-46fc-910e-b274745ea7fc None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:47:38.796526 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c49a98-0f16-46fc-910e-b274745ea7fc None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:47:38.802821 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 695/1401] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:47:38 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:47:38.852732 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf7831-7f22-405b-b3ca-d729f42e2fe0 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:47:38.854000 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-76bf7831-7f22-405b-b3ca-d729f42e2fe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:38.881540 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:38.926343 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6081fef3-1f89-4353-b1e5-441f238e0727 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0daa49b5a35f4a468bb8d0939e7ec20c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:47:38.926964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081fef3-1f89-4353-b1e5-441f238e0727 service swift] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:38.927293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081fef3-1f89-4353-b1e5-441f238e0727 service swift] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:38.927567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6081fef3-1f89-4353-b1e5-441f238e0727 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:38.929238 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-6081fef3-1f89-4353-b1e5-441f238e0727 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:38.932198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:38.997815 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bf7831-7f22-405b-b3ca-d729f42e2fe0 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:47:39.032403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6081fef3-1f89-4353-b1e5-441f238e0727 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:47:39.034003 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6081fef3-1f89-4353-b1e5-441f238e0727 service swift] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:47:39.043999 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 707/1402] 10.222.0.115 () {60 vars in 1389 bytes} [Tue Aug 2 10:47:38 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 02 10:47:39.134586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32884ec-2049-4ae7-9858-730167f1524c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:47:39.143341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32884ec-2049-4ae7-9858-730167f1524c service nova] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:39.143341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32884ec-2049-4ae7-9858-730167f1524c service nova] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:39.143341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32884ec-2049-4ae7-9858-730167f1524c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:39.143341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-a32884ec-2049-4ae7-9858-730167f1524c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:39.157080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:39.230991 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32884ec-2049-4ae7-9858-730167f1524c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:47:39.236832 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32884ec-2049-4ae7-9858-730167f1524c service nova] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:47:39.253083 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 696/1403] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:47:38 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:47:41.681101 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fcdf7-3361-49a5-8173-8799ee4b4222 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:41.681699 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fcdf7-3361-49a5-8173-8799ee4b4222 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:41.681699 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0fcdf7-3361-49a5-8173-8799ee4b4222 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:41.683015 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 708/1404] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:47:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:47:41.690161 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c4dfd-3c1e-45a2-8cc3-35ff45f0f8b8 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:41.690161 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c4dfd-3c1e-45a2-8cc3-35ff45f0f8b8 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:41.690161 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149c4dfd-3c1e-45a2-8cc3-35ff45f0f8b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:41.767470 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-149c4dfd-3c1e-45a2-8cc3-35ff45f0f8b8 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:47:41.768824 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-149c4dfd-3c1e-45a2-8cc3-35ff45f0f8b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:41.875282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:41.882283 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 697/1405] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:47:41 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:47:41.929355 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0fcdf7-3361-49a5-8173-8799ee4b4222 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:47:41.931851 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-0d0fcdf7-3361-49a5-8173-8799ee4b4222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:42.102778 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:42.105934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0fcdf7-3361-49a5-8173-8799ee4b4222 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:47:42.241983 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f24ed11-1ebd-4758-9e45-7550c33bb886 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:47:42.242717 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24ed11-1ebd-4758-9e45-7550c33bb886 service nova] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:42.243099 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24ed11-1ebd-4758-9e45-7550c33bb886 service nova] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:42.243467 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24ed11-1ebd-4758-9e45-7550c33bb886 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:42.245179 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-3f24ed11-1ebd-4758-9e45-7550c33bb886 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:42.359522 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f24ed11-1ebd-4758-9e45-7550c33bb886 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:47:42.361047 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f24ed11-1ebd-4758-9e45-7550c33bb886 service nova] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:47:42.385516 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 709/1406] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:47:41 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:47:44.663549 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b196d0d-b3c5-4c4c-ae01-7b0df36ce606 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:44.664597 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b196d0d-b3c5-4c4c-ae01-7b0df36ce606 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:44.665124 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b196d0d-b3c5-4c4c-ae01-7b0df36ce606 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:44.666995 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 698/1407] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:47:44 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:47:44.676917 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940639c7-5abe-4ea3-b4da-918a7201c23e None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:44.677411 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940639c7-5abe-4ea3-b4da-918a7201c23e None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:44.677495 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940639c7-5abe-4ea3-b4da-918a7201c23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:44.682332 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:44.778281 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-940639c7-5abe-4ea3-b4da-918a7201c23e None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:47:44.779695 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-940639c7-5abe-4ea3-b4da-918a7201c23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:44.898322 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 710/1408] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:47:44 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:47:44.946301 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b196d0d-b3c5-4c4c-ae01-7b0df36ce606 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:47:44.948062 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-9b196d0d-b3c5-4c4c-ae01-7b0df36ce606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:45.131625 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b196d0d-b3c5-4c4c-ae01-7b0df36ce606 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:47:45.247496 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:45.320224 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa2583-6332-4583-a98f-e45be7e5b102 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:47:45.321374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa2583-6332-4583-a98f-e45be7e5b102 service nova] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:47:45.321674 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa2583-6332-4583-a98f-e45be7e5b102 service nova] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:47:45.321939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa2583-6332-4583-a98f-e45be7e5b102 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:47:45.323749 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-b7fa2583-6332-4583-a98f-e45be7e5b102 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:47:45.460889 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fa2583-6332-4583-a98f-e45be7e5b102 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:47:45.462500 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fa2583-6332-4583-a98f-e45be7e5b102 service nova] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:47:45.483760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 699/1409] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:47:44 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:47:54.896658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=64 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:47:55.480838 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=64 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:02.990180 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece6275-8f2a-4176-8832-c95799737283 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:02.990779 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece6275-8f2a-4176-8832-c95799737283 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:02.990779 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fece6275-8f2a-4176-8832-c95799737283 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:02.992200 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 711/1410] 10.222.0.115 () {54 vars in 923 bytes} [Tue Aug 2 10:48:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:48:02.997658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e3946-9602-4d55-b000-1271097ed188 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:02.998186 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e3946-9602-4d55-b000-1271097ed188 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:02.998294 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26e3946-9602-4d55-b000-1271097ed188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:03.078081 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-f26e3946-9602-4d55-b000-1271097ed188 None None] MFA Rules not processed for user `cbc4b962e2a742b08beef7187cc527e5`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:48:03.079272 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f26e3946-9602-4d55-b000-1271097ed188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:03.202896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 700/1411] 10.222.0.115 () {58 vars in 1020 bytes} [Tue Aug 2 10:48:02 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:48:03.219569 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda77d35-150f-4ae6-b92a-283064211322 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:03.219857 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda77d35-150f-4ae6-b92a-283064211322 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:03.220132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda77d35-150f-4ae6-b92a-283064211322 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:03.222389 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 712/1412] 10.222.0.115 () {54 vars in 923 bytes} [Tue Aug 2 10:48:03 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:48:03.227133 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7f971-2b58-4c73-9a5f-a935da8eaef9 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:03.227427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7f971-2b58-4c73-9a5f-a935da8eaef9 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:03.227706 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7f971-2b58-4c73-9a5f-a935da8eaef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:03.304060 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:03.307556 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-11f7f971-2b58-4c73-9a5f-a935da8eaef9 None None] MFA Rules not processed for user `d0478bd540f446ec810e2091c46ac49c`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:48:03.308778 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-11f7f971-2b58-4c73-9a5f-a935da8eaef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:03.437472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 701/1413] 10.222.0.115 () {58 vars in 1020 bytes} [Tue Aug 2 10:48:03 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:48:05.915719 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b9ec4-c001-4300-8476-90fe7dfc48d6 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:05.916269 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b9ec4-c001-4300-8476-90fe7dfc48d6 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:05.916269 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795b9ec4-c001-4300-8476-90fe7dfc48d6 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:05.917624 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 713/1414] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:48:05 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:48:05.923326 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238df88-4efc-4e5f-864c-633f21cc824a None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:05.923791 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238df88-4efc-4e5f-864c-633f21cc824a None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:05.923898 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238df88-4efc-4e5f-864c-633f21cc824a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:05.992248 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-4238df88-4efc-4e5f-864c-633f21cc824a None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:48:05.993484 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-4238df88-4efc-4e5f-864c-633f21cc824a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:06.050039 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:06.092969 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 702/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:48:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:48:06.134893 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b9ec4-c001-4300-8476-90fe7dfc48d6 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:48:06.136586 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-795b9ec4-c001-4300-8476-90fe7dfc48d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:06.283755 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795b9ec4-c001-4300-8476-90fe7dfc48d6 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:48:06.361995 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:06.413396 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af67b4-eebb-4f04-8967-dc0d55a9a7a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:48:06.414084 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af67b4-eebb-4f04-8967-dc0d55a9a7a2 service nova] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:06.414362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af67b4-eebb-4f04-8967-dc0d55a9a7a2 service nova] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:06.414633 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af67b4-eebb-4f04-8967-dc0d55a9a7a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:06.416281 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-36af67b4-eebb-4f04-8967-dc0d55a9a7a2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:06.506105 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af67b4-eebb-4f04-8967-dc0d55a9a7a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:48:06.507747 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af67b4-eebb-4f04-8967-dc0d55a9a7a2 service nova] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:48:06.527480 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 714/1416] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:48:06 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:48:07.433577 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7ac58-1f98-4c99-b1b0-e810985038d9 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:07.434145 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7ac58-1f98-4c99-b1b0-e810985038d9 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:07.434145 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe7ac58-1f98-4c99-b1b0-e810985038d9 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:07.436042 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 703/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:48:07 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:48:07.441222 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c11dd5-f7dc-4417-a465-7aba0a2b8efa None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:07.441668 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c11dd5-f7dc-4417-a465-7aba0a2b8efa None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:07.441787 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c11dd5-f7dc-4417-a465-7aba0a2b8efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:07.522591 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-97c11dd5-f7dc-4417-a465-7aba0a2b8efa None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:48:07.523832 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-97c11dd5-f7dc-4417-a465-7aba0a2b8efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:07.551110 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:07.624834 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 715/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:48:07 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:48:07.777026 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7ac58-1f98-4c99-b1b0-e810985038d9 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:48:07.779827 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-2fe7ac58-1f98-4c99-b1b0-e810985038d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:07.932206 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe7ac58-1f98-4c99-b1b0-e810985038d9 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:48:07.986540 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:08.068268 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256084eb-e708-49cd-bb91-b808eca0c116 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:48:08.068795 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256084eb-e708-49cd-bb91-b808eca0c116 service nova] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:08.069075 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256084eb-e708-49cd-bb91-b808eca0c116 service nova] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:08.069354 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256084eb-e708-49cd-bb91-b808eca0c116 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:08.071021 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-256084eb-e708-49cd-bb91-b808eca0c116 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:08.174680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256084eb-e708-49cd-bb91-b808eca0c116 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:48:08.176486 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256084eb-e708-49cd-bb91-b808eca0c116 service nova] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:48:08.196335 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 704/1419] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:48:07 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:48:09.107102 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075cc7b-e33b-4af8-b418-4cc32f87489d None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:09.107706 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075cc7b-e33b-4af8-b418-4cc32f87489d None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:09.107706 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2075cc7b-e33b-4af8-b418-4cc32f87489d None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:09.108865 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 716/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:48:09 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:48:09.113840 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59da0b1-ba6d-49e4-a98a-14589baf2611 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:09.114235 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59da0b1-ba6d-49e4-a98a-14589baf2611 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:09.114373 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59da0b1-ba6d-49e4-a98a-14589baf2611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:09.177128 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:09.205492 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-e59da0b1-ba6d-49e4-a98a-14589baf2611 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:48:09.206744 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-e59da0b1-ba6d-49e4-a98a-14589baf2611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:09.306783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 705/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:48:09 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:48:09.472467 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075cc7b-e33b-4af8-b418-4cc32f87489d None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:48:09.474013 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-2075cc7b-e33b-4af8-b418-4cc32f87489d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:09.632119 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2075cc7b-e33b-4af8-b418-4cc32f87489d None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:48:09.645067 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:09.749224 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cde5f6b-7f5a-4b73-8594-d501d30f5c38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:48:09.749738 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde5f6b-7f5a-4b73-8594-d501d30f5c38 service nova] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:09.750013 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde5f6b-7f5a-4b73-8594-d501d30f5c38 service nova] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:09.750283 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cde5f6b-7f5a-4b73-8594-d501d30f5c38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:09.751924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-4cde5f6b-7f5a-4b73-8594-d501d30f5c38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:09.841771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde5f6b-7f5a-4b73-8594-d501d30f5c38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:48:09.843393 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cde5f6b-7f5a-4b73-8594-d501d30f5c38 service nova] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:48:09.863084 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 717/1422] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:48:09 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:48:10.731832 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c2aef-8946-41dc-b69b-6a066fcc653c None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:10.732422 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c2aef-8946-41dc-b69b-6a066fcc653c None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:10.732422 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2c2aef-8946-41dc-b69b-6a066fcc653c None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:10.733641 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 706/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:48:10 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:48:10.738684 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2948d4-04a7-4c33-b97b-7b74120109e3 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:10.739153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2948d4-04a7-4c33-b97b-7b74120109e3 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:10.739275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2948d4-04a7-4c33-b97b-7b74120109e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:10.759159 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:10.819517 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-1d2948d4-04a7-4c33-b97b-7b74120109e3 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:48:10.821120 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-1d2948d4-04a7-4c33-b97b-7b74120109e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:10.920422 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 718/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:48:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:48:11.073768 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c2aef-8946-41dc-b69b-6a066fcc653c None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:48:11.075677 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-9e2c2aef-8946-41dc-b69b-6a066fcc653c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:11.190596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:11.227584 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2c2aef-8946-41dc-b69b-6a066fcc653c None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:48:11.345235 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35d5004-30a5-44c4-91fd-327e7eae576c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:48:11.345759 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d5004-30a5-44c4-91fd-327e7eae576c service nova] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:11.346034 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d5004-30a5-44c4-91fd-327e7eae576c service nova] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:11.346303 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35d5004-30a5-44c4-91fd-327e7eae576c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:11.347938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f35d5004-30a5-44c4-91fd-327e7eae576c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:11.431232 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:11.447336 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d5004-30a5-44c4-91fd-327e7eae576c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:48:11.448961 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35d5004-30a5-44c4-91fd-327e7eae576c service nova] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:48:11.469035 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 707/1425] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:48:11 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:48:12.357081 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5457812-50e8-42f3-95cf-1100ad881a21 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:12.357658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5457812-50e8-42f3-95cf-1100ad881a21 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:12.357777 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5457812-50e8-42f3-95cf-1100ad881a21 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:12.359205 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 719/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:48:12 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:48:12.366540 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27472d16-52e8-4bc0-a1b4-15f437a7d9c6 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:12.366952 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27472d16-52e8-4bc0-a1b4-15f437a7d9c6 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:12.367167 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27472d16-52e8-4bc0-a1b4-15f437a7d9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:12.452571 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-27472d16-52e8-4bc0-a1b4-15f437a7d9c6 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:48:12.453786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-27472d16-52e8-4bc0-a1b4-15f437a7d9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:12.554820 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 708/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:48:12 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:48:12.706930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5457812-50e8-42f3-95cf-1100ad881a21 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:48:12.708385 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-a5457812-50e8-42f3-95cf-1100ad881a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:12.795276 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:12.858614 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5457812-50e8-42f3-95cf-1100ad881a21 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:48:12.983860 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05667e0e-de25-498c-a262-eb92bd89711d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:48:12.984367 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05667e0e-de25-498c-a262-eb92bd89711d service nova] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:12.984651 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05667e0e-de25-498c-a262-eb92bd89711d service nova] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:12.984924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05667e0e-de25-498c-a262-eb92bd89711d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:12.986554 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-05667e0e-de25-498c-a262-eb92bd89711d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:13.035026 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:13.077948 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05667e0e-de25-498c-a262-eb92bd89711d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:48:13.079561 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05667e0e-de25-498c-a262-eb92bd89711d service nova] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:48:13.100676 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 720/1428] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:48:12 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:48:13.990030 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d33f9a-bc7d-4c0e-8119-20f4dcebefec None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:13.990610 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d33f9a-bc7d-4c0e-8119-20f4dcebefec None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:13.990610 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d33f9a-bc7d-4c0e-8119-20f4dcebefec None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:13.992053 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 709/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:48:13 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:48:13.997712 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503b469-457e-4b2f-8024-8e910e4ad92f None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:14.003869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503b469-457e-4b2f-8024-8e910e4ad92f None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:14.003869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2503b469-457e-4b2f-8024-8e910e4ad92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:14.088933 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-2503b469-457e-4b2f-8024-8e910e4ad92f None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:48:14.090172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-2503b469-457e-4b2f-8024-8e910e4ad92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:14.198605 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 721/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:48:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:48:14.355200 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d33f9a-bc7d-4c0e-8119-20f4dcebefec None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:48:14.357084 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-44d33f9a-bc7d-4c0e-8119-20f4dcebefec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:14.413115 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:14.536858 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d33f9a-bc7d-4c0e-8119-20f4dcebefec None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:48:14.660009 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b93986-28df-44a9-9e6f-62e2814be263 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:48:14.660625 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b93986-28df-44a9-9e6f-62e2814be263 service nova] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:14.660898 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b93986-28df-44a9-9e6f-62e2814be263 service nova] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:14.661170 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b93986-28df-44a9-9e6f-62e2814be263 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:14.662798 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f5b93986-28df-44a9-9e6f-62e2814be263 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:14.670638 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:14.760383 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b93986-28df-44a9-9e6f-62e2814be263 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:48:14.761896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b93986-28df-44a9-9e6f-62e2814be263 service nova] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:48:14.784588 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 710/1431] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:48:14 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:48:15.642710 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5a30a-fff8-4482-96bc-a016e64e1877 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:15.643221 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5a30a-fff8-4482-96bc-a016e64e1877 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:15.643221 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5a30a-fff8-4482-96bc-a016e64e1877 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:15.644588 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 722/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:48:15 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:48:15.649604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b16680-be58-450b-aad9-6042c3f3b312 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:15.650077 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b16680-be58-450b-aad9-6042c3f3b312 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:15.650211 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b16680-be58-450b-aad9-6042c3f3b312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:15.728396 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-72b16680-be58-450b-aad9-6042c3f3b312 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:48:15.729606 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-72b16680-be58-450b-aad9-6042c3f3b312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:15.806014 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:15.839960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 711/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:48:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:48:15.991198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5a30a-fff8-4482-96bc-a016e64e1877 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:48:15.992578 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-39e5a30a-fff8-4482-96bc-a016e64e1877 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:16.010515 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:16.162297 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5a30a-fff8-4482-96bc-a016e64e1877 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:48:16.256521 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:16.283450 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e6396-2330-4052-8bbc-b2954c1cfb01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:48:16.284073 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e6396-2330-4052-8bbc-b2954c1cfb01 service nova] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:16.284363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e6396-2330-4052-8bbc-b2954c1cfb01 service nova] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:16.284637 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e6396-2330-4052-8bbc-b2954c1cfb01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:16.288056 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-7c0e6396-2330-4052-8bbc-b2954c1cfb01 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:16.397808 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e6396-2330-4052-8bbc-b2954c1cfb01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:48:16.400378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0e6396-2330-4052-8bbc-b2954c1cfb01 service nova] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:48:16.421300 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 723/1434] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:48:15 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:48:17.334976 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea8c2c-dfa5-4d14-925c-fb58cb271dd1 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:17.335614 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea8c2c-dfa5-4d14-925c-fb58cb271dd1 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:17.335614 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea8c2c-dfa5-4d14-925c-fb58cb271dd1 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:17.336928 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 712/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:48:17 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:48:17.342614 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfafc73-6d98-4f2a-a5a1-90c826d18dde None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:17.343192 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfafc73-6d98-4f2a-a5a1-90c826d18dde None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:17.343360 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfafc73-6d98-4f2a-a5a1-90c826d18dde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:17.438408 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-4dfafc73-6d98-4f2a-a5a1-90c826d18dde None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:48:17.439650 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-4dfafc73-6d98-4f2a-a5a1-90c826d18dde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:17.488111 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:17.551491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 724/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:48:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:48:17.712341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea8c2c-dfa5-4d14-925c-fb58cb271dd1 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:48:17.713776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-38ea8c2c-dfa5-4d14-925c-fb58cb271dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:17.889792 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea8c2c-dfa5-4d14-925c-fb58cb271dd1 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:48:17.961886 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:18.023875 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1adddab-4993-4b77-9d3f-6569aad6e343 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:48:18.024498 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1adddab-4993-4b77-9d3f-6569aad6e343 service nova] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:18.024778 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1adddab-4993-4b77-9d3f-6569aad6e343 service nova] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:18.025050 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1adddab-4993-4b77-9d3f-6569aad6e343 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:18.026699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-e1adddab-4993-4b77-9d3f-6569aad6e343 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:18.118067 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1adddab-4993-4b77-9d3f-6569aad6e343 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:48:18.119698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1adddab-4993-4b77-9d3f-6569aad6e343 service nova] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:48:18.138829 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 713/1437] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:48:17 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:48:19.019508 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34f024-46e2-447e-9151-1e99a08617bd None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:19.020357 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34f024-46e2-447e-9151-1e99a08617bd None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:19.020584 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34f024-46e2-447e-9151-1e99a08617bd None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:19.021998 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 725/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:48:19 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:48:19.026742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181022d3-9a01-431a-8ab7-171111b1bb00 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:19.027201 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181022d3-9a01-431a-8ab7-171111b1bb00 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:19.027336 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181022d3-9a01-431a-8ab7-171111b1bb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:19.109790 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-181022d3-9a01-431a-8ab7-171111b1bb00 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:48:19.110983 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-181022d3-9a01-431a-8ab7-171111b1bb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:19.113304 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:19.211906 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 714/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:48:19 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:48:19.367366 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34f024-46e2-447e-9151-1e99a08617bd None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:48:19.368728 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-ce34f024-46e2-447e-9151-1e99a08617bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:19.522008 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce34f024-46e2-447e-9151-1e99a08617bd None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:48:19.548940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:19.639184 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e3e7a-8f92-45a5-8c68-9dfe436fd20a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:48:19.639724 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e3e7a-8f92-45a5-8c68-9dfe436fd20a service nova] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:19.640006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e3e7a-8f92-45a5-8c68-9dfe436fd20a service nova] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:19.640281 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e3e7a-8f92-45a5-8c68-9dfe436fd20a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:19.641936 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-838e3e7a-8f92-45a5-8c68-9dfe436fd20a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:19.738858 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e3e7a-8f92-45a5-8c68-9dfe436fd20a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:48:19.740490 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e3e7a-8f92-45a5-8c68-9dfe436fd20a service nova] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:48:19.761597 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 726/1440] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:48:19 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:48:21.696659 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160eb35-9454-4ba0-a25e-52bad9d501d9 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:21.697234 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160eb35-9454-4ba0-a25e-52bad9d501d9 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:21.697234 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4160eb35-9454-4ba0-a25e-52bad9d501d9 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:21.698565 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 715/1441] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:48:21 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:48:21.704924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63edb5ee-b8c6-4021-8340-64c07c88133c None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:21.705416 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63edb5ee-b8c6-4021-8340-64c07c88133c None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:21.705767 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63edb5ee-b8c6-4021-8340-64c07c88133c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:21.749485 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:21.776697 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-63edb5ee-b8c6-4021-8340-64c07c88133c None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:48:21.778570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-63edb5ee-b8c6-4021-8340-64c07c88133c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:21.882337 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 727/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:48:21 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:48:21.921932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160eb35-9454-4ba0-a25e-52bad9d501d9 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:48:21.923538 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-4160eb35-9454-4ba0-a25e-52bad9d501d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:22.032631 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4160eb35-9454-4ba0-a25e-52bad9d501d9 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:48:22.066763 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:22.115563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6536bc06-fc65-4045-b611-931b664bd267 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:48:22.116190 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536bc06-fc65-4045-b611-931b664bd267 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:22.116513 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536bc06-fc65-4045-b611-931b664bd267 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:22.116790 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6536bc06-fc65-4045-b611-931b664bd267 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:22.118431 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-6536bc06-fc65-4045-b611-931b664bd267 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:22.239682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6536bc06-fc65-4045-b611-931b664bd267 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:48:22.241114 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6536bc06-fc65-4045-b611-931b664bd267 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:48:22.382587 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 716/1443] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:48:21 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:48:24.590642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b3862-f4c4-49c9-bd7b-1e2b2f0e2daf None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:24.591312 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b3862-f4c4-49c9-bd7b-1e2b2f0e2daf None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:24.591438 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b3862-f4c4-49c9-bd7b-1e2b2f0e2daf None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:24.592805 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 728/1444] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:48:24 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:48:24.598003 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4baa23-962c-4f31-a7e2-501cf4a163c3 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:24.598447 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4baa23-962c-4f31-a7e2-501cf4a163c3 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:24.598559 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4baa23-962c-4f31-a7e2-501cf4a163c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:24.671669 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-ab4baa23-962c-4f31-a7e2-501cf4a163c3 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:48:24.672788 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-ab4baa23-962c-4f31-a7e2-501cf4a163c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:24.687004 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:24.774473 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 717/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:48:24 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:48:24.815385 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b3862-f4c4-49c9-bd7b-1e2b2f0e2daf None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:48:24.817154 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-396b3862-f4c4-49c9-bd7b-1e2b2f0e2daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:24.930393 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b3862-f4c4-49c9-bd7b-1e2b2f0e2daf None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:48:24.943033 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:25.006898 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d50b9-2478-4140-ba6c-98d23b5b430f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:48:25.008311 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d50b9-2478-4140-ba6c-98d23b5b430f service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:25.008311 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d50b9-2478-4140-ba6c-98d23b5b430f service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:25.008311 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d50b9-2478-4140-ba6c-98d23b5b430f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:25.009708 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-cd5d50b9-2478-4140-ba6c-98d23b5b430f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:25.128297 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d50b9-2478-4140-ba6c-98d23b5b430f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:48:25.130031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d50b9-2478-4140-ba6c-98d23b5b430f service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:48:25.150275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 729/1446] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:48:24 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:48:27.638872 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c4441-8afd-4b31-9c2a-61ae16aa65a3 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:27.639543 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c4441-8afd-4b31-9c2a-61ae16aa65a3 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:27.639543 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45c4441-8afd-4b31-9c2a-61ae16aa65a3 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:27.640874 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 718/1447] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:48:27 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:48:27.646244 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceeaad5-2f8f-406b-a8a0-bd82047cb354 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:27.646669 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceeaad5-2f8f-406b-a8a0-bd82047cb354 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:27.646812 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceeaad5-2f8f-406b-a8a0-bd82047cb354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:27.697328 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:27.717583 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-fceeaad5-2f8f-406b-a8a0-bd82047cb354 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:48:27.717583 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-fceeaad5-2f8f-406b-a8a0-bd82047cb354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:27.819443 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 730/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:48:27 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:48:27.862340 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c4441-8afd-4b31-9c2a-61ae16aa65a3 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:48:27.863908 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-c45c4441-8afd-4b31-9c2a-61ae16aa65a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:27.970028 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45c4441-8afd-4b31-9c2a-61ae16aa65a3 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:48:28.011864 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:28.046129 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c5659e-61bb-4000-8bf5-9589df953c4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:48:28.046744 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5659e-61bb-4000-8bf5-9589df953c4b service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:28.047020 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5659e-61bb-4000-8bf5-9589df953c4b service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:28.047327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5659e-61bb-4000-8bf5-9589df953c4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:28.048949 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-64c5659e-61bb-4000-8bf5-9589df953c4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:28.160400 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c5659e-61bb-4000-8bf5-9589df953c4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:48:28.161833 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c5659e-61bb-4000-8bf5-9589df953c4b service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:48:28.180462 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 719/1449] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:48:27 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:48:30.406614 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f5a1d-8e14-4477-8a7e-4e839a4d87ac None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:30.407314 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f5a1d-8e14-4477-8a7e-4e839a4d87ac None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:30.407314 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f5a1d-8e14-4477-8a7e-4e839a4d87ac None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:30.408685 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 731/1450] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:48:30 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:48:30.414140 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7d44f-7b38-4904-896e-0ff56c67b399 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:30.414563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7d44f-7b38-4904-896e-0ff56c67b399 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:30.414683 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7d44f-7b38-4904-896e-0ff56c67b399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:30.499242 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-89b7d44f-7b38-4904-896e-0ff56c67b399 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:48:30.500472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-89b7d44f-7b38-4904-896e-0ff56c67b399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:30.520977 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:30.603022 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 720/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:48:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:48:30.644582 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f5a1d-8e14-4477-8a7e-4e839a4d87ac None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:48:30.646112 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-0f2f5a1d-8e14-4477-8a7e-4e839a4d87ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:30.758890 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f5a1d-8e14-4477-8a7e-4e839a4d87ac None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:48:30.780416 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:30.841835 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83fcb2-b1f7-4c75-b85d-12ae14ead997 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:48:30.842463 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83fcb2-b1f7-4c75-b85d-12ae14ead997 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:30.842750 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83fcb2-b1f7-4c75-b85d-12ae14ead997 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:30.843035 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83fcb2-b1f7-4c75-b85d-12ae14ead997 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:30.844715 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-5a83fcb2-b1f7-4c75-b85d-12ae14ead997 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:30.962860 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a83fcb2-b1f7-4c75-b85d-12ae14ead997 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:48:30.964327 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a83fcb2-b1f7-4c75-b85d-12ae14ead997 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:48:30.983456 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 732/1452] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:48:30 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:48:33.100330 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216b203-8801-4f51-beab-1fbd87b5375b None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:33.100847 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216b203-8801-4f51-beab-1fbd87b5375b None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:33.100847 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216b203-8801-4f51-beab-1fbd87b5375b None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:33.102167 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 721/1453] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:48:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:48:33.107877 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb40b4-dff8-4753-ab62-29c314176882 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:33.108332 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb40b4-dff8-4753-ab62-29c314176882 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:33.108440 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb40b4-dff8-4753-ab62-29c314176882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:33.176192 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:33.179350 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-92cb40b4-dff8-4753-ab62-29c314176882 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:48:33.180643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-92cb40b4-dff8-4753-ab62-29c314176882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:33.289909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 733/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:48:33 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:48:33.329610 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216b203-8801-4f51-beab-1fbd87b5375b None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:48:33.331210 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-7216b203-8801-4f51-beab-1fbd87b5375b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:33.436994 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216b203-8801-4f51-beab-1fbd87b5375b None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:48:33.489860 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:33.517889 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13810df-d076-4005-95a3-edb013dfa1c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:48:33.518509 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13810df-d076-4005-95a3-edb013dfa1c2 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:33.518793 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13810df-d076-4005-95a3-edb013dfa1c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:33.519086 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13810df-d076-4005-95a3-edb013dfa1c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:33.520730 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-c13810df-d076-4005-95a3-edb013dfa1c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:33.630900 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13810df-d076-4005-95a3-edb013dfa1c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:48:33.632379 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13810df-d076-4005-95a3-edb013dfa1c2 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:48:33.652678 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 722/1455] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:48:33 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:48:35.807751 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75734809-2127-4a22-9401-2549efb1dbcf None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:35.808604 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75734809-2127-4a22-9401-2549efb1dbcf None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:35.808604 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75734809-2127-4a22-9401-2549efb1dbcf None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:35.810060 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 734/1456] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:48:35 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:48:35.816281 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0015c-709e-4451-9f31-525a2eabeeed None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:35.816765 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0015c-709e-4451-9f31-525a2eabeeed None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:35.816883 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0015c-709e-4451-9f31-525a2eabeeed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:35.885836 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-57e0015c-709e-4451-9f31-525a2eabeeed None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:48:35.887442 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-57e0015c-709e-4451-9f31-525a2eabeeed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:35.916723 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:35.986435 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 723/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:48:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:48:36.026029 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75734809-2127-4a22-9401-2549efb1dbcf None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:48:36.027713 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-75734809-2127-4a22-9401-2549efb1dbcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:36.137842 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75734809-2127-4a22-9401-2549efb1dbcf None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:48:36.166700 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:36.215402 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28620eaf-e638-43c2-bc49-66ff384d76c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:48:36.216020 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28620eaf-e638-43c2-bc49-66ff384d76c0 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:36.216298 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28620eaf-e638-43c2-bc49-66ff384d76c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:36.216568 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28620eaf-e638-43c2-bc49-66ff384d76c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:36.218256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-28620eaf-e638-43c2-bc49-66ff384d76c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:36.346838 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28620eaf-e638-43c2-bc49-66ff384d76c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:48:36.348471 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28620eaf-e638-43c2-bc49-66ff384d76c0 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:48:36.367357 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 735/1458] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:48:36 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:48:38.522478 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da06103-c928-4217-a29a-a0df10b4adca None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:38.523082 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da06103-c928-4217-a29a-a0df10b4adca None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:38.523082 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da06103-c928-4217-a29a-a0df10b4adca None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:38.524458 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 724/1459] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:48:38 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:48:38.530442 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db5065-3365-49d8-b8a0-0132a424e1e0 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:38.530957 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db5065-3365-49d8-b8a0-0132a424e1e0 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:38.531101 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7db5065-3365-49d8-b8a0-0132a424e1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:38.600634 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-f7db5065-3365-49d8-b8a0-0132a424e1e0 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:48:38.601855 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-f7db5065-3365-49d8-b8a0-0132a424e1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:38.610695 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:38.705148 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 736/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:48:38 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:48:38.743551 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da06103-c928-4217-a29a-a0df10b4adca None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:48:38.745074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-1da06103-c928-4217-a29a-a0df10b4adca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:38.856827 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da06103-c928-4217-a29a-a0df10b4adca None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:48:38.917578 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:38.937008 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2739b11f-71cf-4594-9672-0669214aeb80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:48:38.937635 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739b11f-71cf-4594-9672-0669214aeb80 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:38.937917 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739b11f-71cf-4594-9672-0669214aeb80 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:38.938191 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2739b11f-71cf-4594-9672-0669214aeb80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:38.939885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-2739b11f-71cf-4594-9672-0669214aeb80 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:39.064835 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2739b11f-71cf-4594-9672-0669214aeb80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:48:39.067244 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2739b11f-71cf-4594-9672-0669214aeb80 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:48:39.094892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 725/1461] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:48:38 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:48:41.184257 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb0245-82bd-43c4-8071-40233876e371 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:41.184823 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb0245-82bd-43c4-8071-40233876e371 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:41.184823 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb0245-82bd-43c4-8071-40233876e371 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:41.186143 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 737/1462] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:48:41 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:48:41.191800 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf1eb5-57c3-40ad-b0b8-847230569ec2 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:41.192259 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf1eb5-57c3-40ad-b0b8-847230569ec2 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:41.192372 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf1eb5-57c3-40ad-b0b8-847230569ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:41.262609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-addf1eb5-57c3-40ad-b0b8-847230569ec2 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:48:41.263874 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-addf1eb5-57c3-40ad-b0b8-847230569ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:41.303065 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:41.363496 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 726/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:48:41 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:48:41.400563 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb0245-82bd-43c4-8071-40233876e371 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:48:41.402772 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-9ceb0245-82bd-43c4-8071-40233876e371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:41.513372 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb0245-82bd-43c4-8071-40233876e371 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:48:41.553491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:41.591722 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5554c93-380b-433f-9c3c-627af24cc921 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:48:41.592346 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5554c93-380b-433f-9c3c-627af24cc921 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:41.592626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5554c93-380b-433f-9c3c-627af24cc921 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:41.592897 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5554c93-380b-433f-9c3c-627af24cc921 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:41.594549 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-f5554c93-380b-433f-9c3c-627af24cc921 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:41.706584 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5554c93-380b-433f-9c3c-627af24cc921 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:48:41.708188 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5554c93-380b-433f-9c3c-627af24cc921 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:48:41.727203 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 738/1464] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:48:41 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:48:43.794637 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b8870-7479-450c-af7e-ac8f1ad9d1a9 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:43.795205 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b8870-7479-450c-af7e-ac8f1ad9d1a9 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:43.795205 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b8870-7479-450c-af7e-ac8f1ad9d1a9 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:43.796588 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 727/1465] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:48:43 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:48:43.802122 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92e5bf-945d-47db-961c-3290f72a8fa9 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:43.802587 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92e5bf-945d-47db-961c-3290f72a8fa9 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:43.802705 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92e5bf-945d-47db-961c-3290f72a8fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:43.872643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-4a92e5bf-945d-47db-961c-3290f72a8fa9 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:48:43.873916 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-4a92e5bf-945d-47db-961c-3290f72a8fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:43.892880 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:43.975197 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 739/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:48:43 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:48:44.012784 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b8870-7479-450c-af7e-ac8f1ad9d1a9 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:48:44.014271 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-4c1b8870-7479-450c-af7e-ac8f1ad9d1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:44.122923 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b8870-7479-450c-af7e-ac8f1ad9d1a9 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:48:44.191790 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:44.199465 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc57d24-29f5-438c-aff5-f377760ed62f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:48:44.200086 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc57d24-29f5-438c-aff5-f377760ed62f service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:44.200369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc57d24-29f5-438c-aff5-f377760ed62f service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:44.200648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc57d24-29f5-438c-aff5-f377760ed62f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:44.202367 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-abc57d24-29f5-438c-aff5-f377760ed62f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:44.317599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc57d24-29f5-438c-aff5-f377760ed62f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:48:44.319032 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc57d24-29f5-438c-aff5-f377760ed62f service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:48:44.337905 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 728/1467] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:48:44 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:48:46.412210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614b7d6-8df8-4b1b-9249-5c114cb7ebe0 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:46.412745 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614b7d6-8df8-4b1b-9249-5c114cb7ebe0 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:46.413162 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3614b7d6-8df8-4b1b-9249-5c114cb7ebe0 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:46.415594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 740/1468] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:48:46 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:48:46.421919 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab1cf6-bf9a-4e86-9470-e315d9c20694 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:46.422368 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab1cf6-bf9a-4e86-9470-e315d9c20694 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:46.422478 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ab1cf6-bf9a-4e86-9470-e315d9c20694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:46.490686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-88ab1cf6-bf9a-4e86-9470-e315d9c20694 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:48:46.491914 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-88ab1cf6-bf9a-4e86-9470-e315d9c20694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:46.540353 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:46.590522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 729/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:48:46 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:48:46.627610 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614b7d6-8df8-4b1b-9249-5c114cb7ebe0 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:48:46.629115 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-3614b7d6-8df8-4b1b-9249-5c114cb7ebe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:46.735510 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3614b7d6-8df8-4b1b-9249-5c114cb7ebe0 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:48:46.788436 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:46.815358 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed609f1b-f5fd-438b-8e1b-8eb035d9c662 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:48:46.815974 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed609f1b-f5fd-438b-8e1b-8eb035d9c662 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:48:46.816378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed609f1b-f5fd-438b-8e1b-8eb035d9c662 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:48:46.816555 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed609f1b-f5fd-438b-8e1b-8eb035d9c662 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:48:46.818217 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-ed609f1b-f5fd-438b-8e1b-8eb035d9c662 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:48:46.935323 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed609f1b-f5fd-438b-8e1b-8eb035d9c662 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:48:46.936763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed609f1b-f5fd-438b-8e1b-8eb035d9c662 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:48:46.955081 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 741/1470] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:48:46 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:48:56.589126 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=22 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:48:56.952768 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=66 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:49:06.771013 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481375b-239d-4dd9-a56b-7732139e9489 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:06.771585 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481375b-239d-4dd9-a56b-7732139e9489 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:06.771585 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481375b-239d-4dd9-a56b-7732139e9489 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:06.773003 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 730/1471] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:49:06 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:49:06.779707 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d6599-36b7-4bb9-8466-4269c8c358ea None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:06.780215 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d6599-36b7-4bb9-8466-4269c8c358ea None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:06.780331 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d6599-36b7-4bb9-8466-4269c8c358ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:06.847773 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-1b0d6599-36b7-4bb9-8466-4269c8c358ea None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:49:06.848980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-1b0d6599-36b7-4bb9-8466-4269c8c358ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:06.949257 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 742/1472] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:49:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:49:07.034833 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78e0e0-89d9-435a-81a6-d13f250fa71c None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:07.035148 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78e0e0-89d9-435a-81a6-d13f250fa71c None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:07.035421 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a78e0e0-89d9-435a-81a6-d13f250fa71c None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:07.036762 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 731/1473] 10.222.0.115 () {54 vars in 971 bytes} [Tue Aug 2 10:49:07 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:49:07.042581 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce721f0-d6eb-47fc-ab7e-22a91f3006e9 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:07.042869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce721f0-d6eb-47fc-ab7e-22a91f3006e9 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:07.043180 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce721f0-d6eb-47fc-ab7e-22a91f3006e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:07.119025 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-0ce721f0-d6eb-47fc-ab7e-22a91f3006e9 None None] MFA Rules not processed for user `e358fd7c1b3c43d7ac89fb3562678266`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:49:07.120268 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-0ce721f0-d6eb-47fc-ab7e-22a91f3006e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:07.140534 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:49:07.199644 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 743/1474] 10.222.0.115 () {58 vars in 1068 bytes} [Tue Aug 2 10:49:07 2022] POST /identity/v3/auth/tokens => generated 3130 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:49:07.207405 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78e0e0-89d9-435a-81a6-d13f250fa71c None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:49:07.208888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-6a78e0e0-89d9-435a-81a6-d13f250fa71c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:07.314108 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a78e0e0-89d9-435a-81a6-d13f250fa71c None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:49:07.388420 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70612054-73a8-4047-a690-9c92c98e34c6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e358fd7c1b3c43d7ac89fb3562678266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:49:07.389037 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70612054-73a8-4047-a690-9c92c98e34c6 service glance] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:07.389340 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70612054-73a8-4047-a690-9c92c98e34c6 service glance] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:07.389615 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70612054-73a8-4047-a690-9c92c98e34c6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:07.391278 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-70612054-73a8-4047-a690-9c92c98e34c6 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:07.445925 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:49:07.502420 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70612054-73a8-4047-a690-9c92c98e34c6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:49:07.504369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70612054-73a8-4047-a690-9c92c98e34c6 service glance] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:49:07.523087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 732/1475] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:49:07 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:49:09.630255 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f134ad8-bccf-43b5-80e4-65881b278166 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:09.630831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f134ad8-bccf-43b5-80e4-65881b278166 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:09.630831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f134ad8-bccf-43b5-80e4-65881b278166 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:09.632253 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 744/1476] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:49:09 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:49:09.637788 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5245aff-62a1-4488-af61-93eb1192f31d None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:09.638238 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5245aff-62a1-4488-af61-93eb1192f31d None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:09.638345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5245aff-62a1-4488-af61-93eb1192f31d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:09.707218 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-b5245aff-62a1-4488-af61-93eb1192f31d None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:49:09.708367 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-b5245aff-62a1-4488-af61-93eb1192f31d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:09.807446 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 733/1477] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:49:09 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:49:09.831802 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f134ad8-bccf-43b5-80e4-65881b278166 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:49:09.833375 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-4f134ad8-bccf-43b5-80e4-65881b278166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:09.963686 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f134ad8-bccf-43b5-80e4-65881b278166 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:49:10.015056 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:49:10.061590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0980415d-02ae-4eea-aa64-6a246febd784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:49:10.062224 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980415d-02ae-4eea-aa64-6a246febd784 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:10.062498 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980415d-02ae-4eea-aa64-6a246febd784 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:10.062765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980415d-02ae-4eea-aa64-6a246febd784 admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:10.063495 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0980415d-02ae-4eea-aa64-6a246febd784 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:49:10.065263 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: WARNING py.warnings [None req-0980415d-02ae-4eea-aa64-6a246febd784 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:49:10.065263 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: warnings.warn(msg) Aug 02 10:49:10.065263 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: Aug 02 10:49:10.066483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0980415d-02ae-4eea-aa64-6a246febd784 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:49:10.086806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 745/1478] 10.222.0.115 () {56 vars in 1113 bytes} [Tue Aug 2 10:49:09 2022] GET /identity/v3/projects => generated 2562 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:49:12.142053 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee5d9d-7e5e-4248-a649-e99d54a1c74b None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:12.142765 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee5d9d-7e5e-4248-a649-e99d54a1c74b None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:12.142765 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee5d9d-7e5e-4248-a649-e99d54a1c74b None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:12.144025 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 734/1479] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:49:12 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:49:12.149226 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc351d0-b46b-49a2-a532-182e0e86c389 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:12.149746 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc351d0-b46b-49a2-a532-182e0e86c389 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:12.149881 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc351d0-b46b-49a2-a532-182e0e86c389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:12.217871 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-0cc351d0-b46b-49a2-a532-182e0e86c389 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:49:12.219450 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-0cc351d0-b46b-49a2-a532-182e0e86c389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:12.321673 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 746/1480] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:49:12 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:49:12.364905 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee5d9d-7e5e-4248-a649-e99d54a1c74b None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:49:12.366520 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-6dee5d9d-7e5e-4248-a649-e99d54a1c74b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:12.377442 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:49:12.533007 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee5d9d-7e5e-4248-a649-e99d54a1c74b None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:49:12.616778 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:49:12.651460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013735b1-e0bc-45d8-9b8e-daefde98f3ca service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:49:12.652075 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013735b1-e0bc-45d8-9b8e-daefde98f3ca service nova] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:12.652352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013735b1-e0bc-45d8-9b8e-daefde98f3ca service nova] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:12.652623 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013735b1-e0bc-45d8-9b8e-daefde98f3ca service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:12.654250 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-013735b1-e0bc-45d8-9b8e-daefde98f3ca service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:12.747334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013735b1-e0bc-45d8-9b8e-daefde98f3ca service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:49:12.748967 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013735b1-e0bc-45d8-9b8e-daefde98f3ca service nova] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:49:12.766967 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 735/1481] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:49:12 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:49:15.007555 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370a2e1-9258-4092-b707-52f8c1970832 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:15.008134 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370a2e1-9258-4092-b707-52f8c1970832 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:15.008134 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6370a2e1-9258-4092-b707-52f8c1970832 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:15.009505 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 747/1482] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:49:15 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:49:15.015097 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c61a80-a442-4f8b-b340-6ca161c39b88 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:15.015581 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c61a80-a442-4f8b-b340-6ca161c39b88 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:15.015708 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c61a80-a442-4f8b-b340-6ca161c39b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:15.090756 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-25c61a80-a442-4f8b-b340-6ca161c39b88 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:49:15.092000 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-25c61a80-a442-4f8b-b340-6ca161c39b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:15.141568 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:49:15.205961 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 736/1483] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:49:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:49:15.242518 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370a2e1-9258-4092-b707-52f8c1970832 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:49:15.244831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-6370a2e1-9258-4092-b707-52f8c1970832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:15.270369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:49:15.354324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6370a2e1-9258-4092-b707-52f8c1970832 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:49:15.435025 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb30d8-5b80-4ec7-9dc1-1c3ee6d0d871 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:49:15.435613 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb30d8-5b80-4ec7-9dc1-1c3ee6d0d871 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:15.435902 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb30d8-5b80-4ec7-9dc1-1c3ee6d0d871 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:15.436181 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb30d8-5b80-4ec7-9dc1-1c3ee6d0d871 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:15.437872 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-14bb30d8-5b80-4ec7-9dc1-1c3ee6d0d871 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:15.518184 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:49:15.560186 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb30d8-5b80-4ec7-9dc1-1c3ee6d0d871 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:49:15.561555 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb30d8-5b80-4ec7-9dc1-1c3ee6d0d871 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:49:15.580599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 748/1484] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:49:15 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:49:17.668391 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017b5d4-4ab0-4072-8743-4cdf6a384cd7 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:17.669172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017b5d4-4ab0-4072-8743-4cdf6a384cd7 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:17.669172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5017b5d4-4ab0-4072-8743-4cdf6a384cd7 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:17.670373 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 737/1485] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:49:17 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:49:17.675852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f8f65-6e68-43d4-a325-b574a7f84987 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:17.676283 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f8f65-6e68-43d4-a325-b574a7f84987 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:17.676442 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f8f65-6e68-43d4-a325-b574a7f84987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:17.753530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-548f8f65-6e68-43d4-a325-b574a7f84987 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:49:17.754623 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-548f8f65-6e68-43d4-a325-b574a7f84987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:17.855812 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 749/1486] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:49:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:49:17.894690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5017b5d4-4ab0-4072-8743-4cdf6a384cd7 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:49:17.896242 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-5017b5d4-4ab0-4072-8743-4cdf6a384cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:18.004535 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5017b5d4-4ab0-4072-8743-4cdf6a384cd7 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:49:18.081275 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:49:18.083151 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bbf6f0-91bd-4d83-bb82-72cca7d84131 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:49:18.083799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbf6f0-91bd-4d83-bb82-72cca7d84131 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:18.084082 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbf6f0-91bd-4d83-bb82-72cca7d84131 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:18.084353 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bbf6f0-91bd-4d83-bb82-72cca7d84131 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:18.086023 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-48bbf6f0-91bd-4d83-bb82-72cca7d84131 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:18.197199 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bbf6f0-91bd-4d83-bb82-72cca7d84131 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:49:18.199237 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bbf6f0-91bd-4d83-bb82-72cca7d84131 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:49:18.218443 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 738/1487] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:49:17 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:49:20.440196 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebac593-6feb-48b8-92e1-0b71673e2d21 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:20.440781 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebac593-6feb-48b8-92e1-0b71673e2d21 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:20.440781 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebac593-6feb-48b8-92e1-0b71673e2d21 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:20.442118 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 750/1488] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:49:20 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:49:20.447606 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80f2ac-3c37-4494-95da-eada3890337d None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:20.448059 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80f2ac-3c37-4494-95da-eada3890337d None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:20.448163 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e80f2ac-3c37-4494-95da-eada3890337d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:20.515989 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-3e80f2ac-3c37-4494-95da-eada3890337d None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:49:20.517210 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-3e80f2ac-3c37-4494-95da-eada3890337d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:20.571495 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:49:20.617666 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 739/1489] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:49:20 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:49:20.659345 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebac593-6feb-48b8-92e1-0b71673e2d21 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:49:20.660873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-bebac593-6feb-48b8-92e1-0b71673e2d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:20.699867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:49:20.770828 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebac593-6feb-48b8-92e1-0b71673e2d21 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:49:20.847290 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f535260d-b0f8-4e41-8e19-d3503668ed4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:49:20.847929 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535260d-b0f8-4e41-8e19-d3503668ed4e service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:20.848219 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535260d-b0f8-4e41-8e19-d3503668ed4e service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:20.848502 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f535260d-b0f8-4e41-8e19-d3503668ed4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:20.850174 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-f535260d-b0f8-4e41-8e19-d3503668ed4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:20.934719 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:49:20.965141 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535260d-b0f8-4e41-8e19-d3503668ed4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:49:20.966568 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f535260d-b0f8-4e41-8e19-d3503668ed4e service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:49:20.984760 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 751/1490] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:49:20 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:49:23.492195 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8722ab-50b4-4458-bb47-b823f79a0eaa None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:23.492754 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8722ab-50b4-4458-bb47-b823f79a0eaa None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:23.492754 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8722ab-50b4-4458-bb47-b823f79a0eaa None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:23.494782 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 740/1491] 10.222.0.115 () {54 vars in 931 bytes} [Tue Aug 2 10:49:23 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:49:23.500112 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29502e7-367e-4582-8ae8-fcbe5eae7af9 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:23.500564 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29502e7-367e-4582-8ae8-fcbe5eae7af9 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:23.500695 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29502e7-367e-4582-8ae8-fcbe5eae7af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:23.568286 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-d29502e7-367e-4582-8ae8-fcbe5eae7af9 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:49:23.569508 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-d29502e7-367e-4582-8ae8-fcbe5eae7af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:23.669628 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 752/1492] 10.222.0.115 () {58 vars in 1028 bytes} [Tue Aug 2 10:49:23 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:49:23.709664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8722ab-50b4-4458-bb47-b823f79a0eaa None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:49:23.711251 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-3c8722ab-50b4-4458-bb47-b823f79a0eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:23.816260 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8722ab-50b4-4458-bb47-b823f79a0eaa None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:49:23.890893 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1550ff7a-f3ea-4d42-9f21-5e5f8ad47244 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:49:23.891647 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550ff7a-f3ea-4d42-9f21-5e5f8ad47244 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:49:23.891885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550ff7a-f3ea-4d42-9f21-5e5f8ad47244 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:49:23.892147 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1550ff7a-f3ea-4d42-9f21-5e5f8ad47244 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:49:23.893802 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-1550ff7a-f3ea-4d42-9f21-5e5f8ad47244 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:49:23.904175 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:49:24.008259 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1550ff7a-f3ea-4d42-9f21-5e5f8ad47244 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:49:24.009637 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1550ff7a-f3ea-4d42-9f21-5e5f8ad47244 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:49:24.029168 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 741/1493] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:49:23 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:49:33.667768 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=82 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:49:34.026654 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=50 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:51:00.850657 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfab70c-8804-46bd-a9e3-b08ede1b7a04 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:51:00.851369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfab70c-8804-46bd-a9e3-b08ede1b7a04 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:51:00.851369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfab70c-8804-46bd-a9e3-b08ede1b7a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:51:00.942496 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-3bfab70c-8804-46bd-a9e3-b08ede1b7a04 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:51:00.944398 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-3bfab70c-8804-46bd-a9e3-b08ede1b7a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:51:01.059670 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 753/1494] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:51:00 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:51:01.068196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d98b9-cd8a-4d9f-867f-6c4b4715faa0 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:51:01.068741 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d98b9-cd8a-4d9f-867f-6c4b4715faa0 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:51:01.068741 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52d98b9-cd8a-4d9f-867f-6c4b4715faa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:51:01.148601 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-f52d98b9-cd8a-4d9f-867f-6c4b4715faa0 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:51:01.149758 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f52d98b9-cd8a-4d9f-867f-6c4b4715faa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:51:01.247535 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 742/1495] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:51:01 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:51:01.252279 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfab70c-8804-46bd-a9e3-b08ede1b7a04 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:51:01.253523 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-3bfab70c-8804-46bd-a9e3-b08ede1b7a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:51:01.265980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfab70c-8804-46bd-a9e3-b08ede1b7a04 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:51:01.329657 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:51:01.367274 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e4489f-874f-49a5-b535-24b55d6fa2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:51:01.367888 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4489f-874f-49a5-b535-24b55d6fa2e8 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:51:01.368171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4489f-874f-49a5-b535-24b55d6fa2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:51:01.368449 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e4489f-874f-49a5-b535-24b55d6fa2e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:51:01.369142 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e4489f-874f-49a5-b535-24b55d6fa2e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:51:01.369451 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e4489f-874f-49a5-b535-24b55d6fa2e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:51:01.370470 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: WARNING py.warnings [None req-a3e4489f-874f-49a5-b535-24b55d6fa2e8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:51:01.370470 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: warnings.warn(msg) Aug 02 10:51:01.370470 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: Aug 02 10:51:01.371381 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e4489f-874f-49a5-b535-24b55d6fa2e8 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:51:01.383548 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 754/1496] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:51:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:51:01.391749 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d98b9-cd8a-4d9f-867f-6c4b4715faa0 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:51:01.402885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52d98b9-cd8a-4d9f-867f-6c4b4715faa0 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:51:01.466759 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:51:01.501669 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f588e-8b01-47c4-9eb5-6fe88c10c0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:51:01.502521 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f588e-8b01-47c4-9eb5-6fe88c10c0de admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:51:01.502844 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f588e-8b01-47c4-9eb5-6fe88c10c0de admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:51:01.503423 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f588e-8b01-47c4-9eb5-6fe88c10c0de admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:51:01.504102 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634f588e-8b01-47c4-9eb5-6fe88c10c0de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:51:01.506123 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: WARNING py.warnings [None req-634f588e-8b01-47c4-9eb5-6fe88c10c0de admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:51:01.506123 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: warnings.warn(msg) Aug 02 10:51:01.506123 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: Aug 02 10:51:01.507589 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634f588e-8b01-47c4-9eb5-6fe88c10c0de admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:51:01.648439 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 743/1497] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:51:01 2022] POST /identity/v3/projects => generated 378 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:51:01.652769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e4489f-874f-49a5-b535-24b55d6fa2e8 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:51:01.654346 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-a3e4489f-874f-49a5-b535-24b55d6fa2e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:51:01.688554 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e4489f-874f-49a5-b535-24b55d6fa2e8 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:51:01.783071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c6360-c00d-464b-8b0a-4398dd785582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:51:01.783717 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c6360-c00d-464b-8b0a-4398dd785582 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:51:01.784001 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c6360-c00d-464b-8b0a-4398dd785582 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:51:01.784278 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c6360-c00d-464b-8b0a-4398dd785582 admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:51:01.784942 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5c6360-c00d-464b-8b0a-4398dd785582 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:51:01.786573 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: WARNING py.warnings [None req-1c5c6360-c00d-464b-8b0a-4398dd785582 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:51:01.786573 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: warnings.warn(msg) Aug 02 10:51:01.786573 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: Aug 02 10:51:01.787918 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5c6360-c00d-464b-8b0a-4398dd785582 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:51:01.892034 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 755/1498] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:51:01 2022] POST /identity/v3/users => generated 290 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:51:01.897297 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f588e-8b01-47c4-9eb5-6fe88c10c0de admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:51:01.898857 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-634f588e-8b01-47c4-9eb5-6fe88c10c0de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:51:01.964720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f588e-8b01-47c4-9eb5-6fe88c10c0de admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:51:02.048556 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:51:02.058885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f8855-cd02-4259-aa3e-c43e26d6f19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:51:02.060686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f8855-cd02-4259-aa3e-c43e26d6f19e admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:51:02.060686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f8855-cd02-4259-aa3e-c43e26d6f19e admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:51:02.060686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107f8855-cd02-4259-aa3e-c43e26d6f19e admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:51:02.061055 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107f8855-cd02-4259-aa3e-c43e26d6f19e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:51:02.061882 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: WARNING py.warnings [None req-107f8855-cd02-4259-aa3e-c43e26d6f19e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:51:02.061882 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: warnings.warn(msg) Aug 02 10:51:02.061882 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: Aug 02 10:51:02.062771 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107f8855-cd02-4259-aa3e-c43e26d6f19e admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:51:02.076076 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 744/1499] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:51:01 2022] GET /identity/v3/roles => generated 1427 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:51:02.080277 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c6360-c00d-464b-8b0a-4398dd785582 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:51:02.081804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-1c5c6360-c00d-464b-8b0a-4398dd785582 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:51:02.093878 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c6360-c00d-464b-8b0a-4398dd785582 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:51:02.119072 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:51:02.189908 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92510579-1b6f-4d48-b13d-6956a85785c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:51:02.190548 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92510579-1b6f-4d48-b13d-6956a85785c9 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:51:02.190831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92510579-1b6f-4d48-b13d-6956a85785c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:51:02.191141 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92510579-1b6f-4d48-b13d-6956a85785c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:51:02.191833 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92510579-1b6f-4d48-b13d-6956a85785c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:51:02.193398 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: WARNING py.warnings [None req-92510579-1b6f-4d48-b13d-6956a85785c9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:51:02.193398 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: warnings.warn(msg) Aug 02 10:51:02.193398 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: Aug 02 10:51:02.194286 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92510579-1b6f-4d48-b13d-6956a85785c9 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:51:02.206730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 756/1500] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:51:02 2022] GET /identity/v3/roles => generated 1427 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:51:02.211824 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f8855-cd02-4259-aa3e-c43e26d6f19e admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:51:02.213360 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-107f8855-cd02-4259-aa3e-c43e26d6f19e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:51:02.247674 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107f8855-cd02-4259-aa3e-c43e26d6f19e admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:51:02.343733 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02d1239-bca3-4a7d-8b2f-97bd11da1391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:51:02.344465 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d1239-bca3-4a7d-8b2f-97bd11da1391 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:51:02.344745 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d1239-bca3-4a7d-8b2f-97bd11da1391 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:51:02.345015 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02d1239-bca3-4a7d-8b2f-97bd11da1391 admin admin] PATH_INFO: `/v3/projects/b2dee76b1e864cff867db2ae405abfe3/users/67660125b7e244efb59c1c46cc5aa1cc/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:51:02.367976 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d1239-bca3-4a7d-8b2f-97bd11da1391 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2dee76b1e864cff867db2ae405abfe3, user_id=67660125b7e244efb59c1c46cc5aa1cc, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:51:02.369017 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: WARNING py.warnings [None req-f02d1239-bca3-4a7d-8b2f-97bd11da1391 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:51:02.369017 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: warnings.warn(msg) Aug 02 10:51:02.369017 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: Aug 02 10:51:02.370961 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02d1239-bca3-4a7d-8b2f-97bd11da1391 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:51:02.459104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 745/1501] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:51:02 2022] PUT /identity/v3/projects/b2dee76b1e864cff867db2ae405abfe3/users/67660125b7e244efb59c1c46cc5aa1cc/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:51:02.472026 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab48d58-16f7-400a-be3e-b7738b971247 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:51:02.478124 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab48d58-16f7-400a-be3e-b7738b971247 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:51:02.478124 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab48d58-16f7-400a-be3e-b7738b971247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:51:02.533205 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-2ab48d58-16f7-400a-be3e-b7738b971247 None None] MFA Rules not processed for user `67660125b7e244efb59c1c46cc5aa1cc`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:51:02.537227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-2ab48d58-16f7-400a-be3e-b7738b971247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:51:02.613133 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 757/1502] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:51:02 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:51:02.627014 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfeb93-711a-4f2e-a729-c79a589b0b01 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:51:02.627340 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfeb93-711a-4f2e-a729-c79a589b0b01 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:51:02.627620 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfeb93-711a-4f2e-a729-c79a589b0b01 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:51:02.629022 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 746/1503] 10.222.0.115 () {54 vars in 969 bytes} [Tue Aug 2 10:51:02 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:51:02.634010 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cff123-07c3-43cb-a0b4-113ece21f3c6 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:51:02.634296 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cff123-07c3-43cb-a0b4-113ece21f3c6 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:51:02.634569 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cff123-07c3-43cb-a0b4-113ece21f3c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:51:02.665239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:51:02.714434 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-a7cff123-07c3-43cb-a0b4-113ece21f3c6 None None] MFA Rules not processed for user `cc9ab3c9075b4a8385ef634dc0b1625e`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:51:02.715695 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-a7cff123-07c3-43cb-a0b4-113ece21f3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:51:02.790673 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 758/1504] 10.222.0.115 () {58 vars in 1066 bytes} [Tue Aug 2 10:51:02 2022] POST /identity/v3/auth/tokens => generated 3130 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:51:02.798989 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfeb93-711a-4f2e-a729-c79a589b0b01 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:51:02.800517 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-7bbfeb93-711a-4f2e-a729-c79a589b0b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:51:02.851542 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:51:02.905418 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfeb93-711a-4f2e-a729-c79a589b0b01 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:51:02.984241 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd09c5-f51a-46a9-bde7-1c9f395dd525 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:51:02.984851 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd09c5-f51a-46a9-bde7-1c9f395dd525 service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:51:02.985126 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd09c5-f51a-46a9-bde7-1c9f395dd525 service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:51:02.985395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cd09c5-f51a-46a9-bde7-1c9f395dd525 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:51:02.987012 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f3cd09c5-f51a-46a9-bde7-1c9f395dd525 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:51:03.087348 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cd09c5-f51a-46a9-bde7-1c9f395dd525 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:51:03.088867 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cd09c5-f51a-46a9-bde7-1c9f395dd525 service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:51:03.092213 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:51:03.106253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 747/1505] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:51:02 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:51:03.153931 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf97fda-4996-45f0-9134-988996d6bea1 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:51:03.154347 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf97fda-4996-45f0-9134-988996d6bea1 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:51:03.154463 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf97fda-4996-45f0-9134-988996d6bea1 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:51:03.155969 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 759/1506] 10.222.0.115 () {52 vars in 835 bytes} [Tue Aug 2 10:51:03 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:51:03.159475 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd09c5-f51a-46a9-bde7-1c9f395dd525 service cinder] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:51:03.160679 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f3cd09c5-f51a-46a9-bde7-1c9f395dd525 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:51:03.262617 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cd09c5-f51a-46a9-bde7-1c9f395dd525 service cinder] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:51:03.360110 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce994e7-52c4-43fe-9753-4eba43b48ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:51:03.360662 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce994e7-52c4-43fe-9753-4eba43b48ca2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:51:03.360942 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce994e7-52c4-43fe-9753-4eba43b48ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:51:03.361217 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce994e7-52c4-43fe-9753-4eba43b48ca2 admin admin] PATH_INFO: `/v3/users/67660125b7e244efb59c1c46cc5aa1cc` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:51:03.374029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce994e7-52c4-43fe-9753-4eba43b48ca2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67660125b7e244efb59c1c46cc5aa1cc)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:51:03.375074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: WARNING py.warnings [None req-2ce994e7-52c4-43fe-9753-4eba43b48ca2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:51:03.375074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: warnings.warn(msg) Aug 02 10:51:03.375074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: Aug 02 10:51:03.376283 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce994e7-52c4-43fe-9753-4eba43b48ca2 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:51:03.386537 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:51:03.558849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2ce994e7-52c4-43fe-9753-4eba43b48ca2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67660125b7e244efb59c1c46cc5aa1cc'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:51:03.572738 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2ce994e7-52c4-43fe-9753-4eba43b48ca2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67660125b7e244efb59c1c46cc5aa1cc'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:51:03.573295 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2ce994e7-52c4-43fe-9753-4eba43b48ca2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67660125b7e244efb59c1c46cc5aa1cc'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:51:03.621911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2ce994e7-52c4-43fe-9753-4eba43b48ca2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67660125b7e244efb59c1c46cc5aa1cc'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:51:03.646665 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2ce994e7-52c4-43fe-9753-4eba43b48ca2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67660125b7e244efb59c1c46cc5aa1cc'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:51:03.652370 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 748/1507] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:51:03 2022] DELETE /identity/v3/users/67660125b7e244efb59c1c46cc5aa1cc => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:51:03.657211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962d9a3-b09c-4c0f-bfa3-63e91450dab2 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:51:03.657497 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962d9a3-b09c-4c0f-bfa3-63e91450dab2 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:51:03.657769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9962d9a3-b09c-4c0f-bfa3-63e91450dab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:51:03.716357 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-9962d9a3-b09c-4c0f-bfa3-63e91450dab2 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:51:03.717650 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-9962d9a3-b09c-4c0f-bfa3-63e91450dab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:51:03.794657 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:51:03.807189 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 760/1508] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:51:03 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:51:03.817017 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce994e7-52c4-43fe-9753-4eba43b48ca2 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:51:03.818276 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-2ce994e7-52c4-43fe-9753-4eba43b48ca2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:51:03.930567 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce994e7-52c4-43fe-9753-4eba43b48ca2 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:51:03.987532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:51:04.007779 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb834b2-7e87-4456-86f6-9e48be0671a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:51:04.008418 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb834b2-7e87-4456-86f6-9e48be0671a6 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:51:04.008709 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb834b2-7e87-4456-86f6-9e48be0671a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:51:04.008987 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb834b2-7e87-4456-86f6-9e48be0671a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:51:04.010820 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-abb834b2-7e87-4456-86f6-9e48be0671a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:51:04.120388 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb834b2-7e87-4456-86f6-9e48be0671a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:51:04.121848 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb834b2-7e87-4456-86f6-9e48be0671a6 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:51:04.140726 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 749/1509] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:51:03 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:51:04.664501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9962d9a3-b09c-4c0f-bfa3-63e91450dab2 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:51:04.665841 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-9962d9a3-b09c-4c0f-bfa3-63e91450dab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:51:04.680664 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9962d9a3-b09c-4c0f-bfa3-63e91450dab2 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:51:04.783722 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73e04b1-cb85-4185-ade1-dd0f4f862b32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:51:04.784362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e04b1-cb85-4185-ade1-dd0f4f862b32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:51:04.784646 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e04b1-cb85-4185-ade1-dd0f4f862b32 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:51:04.784919 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73e04b1-cb85-4185-ade1-dd0f4f862b32 admin admin] PATH_INFO: `/v3/projects/b2dee76b1e864cff867db2ae405abfe3` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:51:04.795618 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e04b1-cb85-4185-ade1-dd0f4f862b32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2dee76b1e864cff867db2ae405abfe3)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:51:04.796647 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: WARNING py.warnings [None req-c73e04b1-cb85-4185-ade1-dd0f4f862b32 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:51:04.796647 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: warnings.warn(msg) Aug 02 10:51:04.796647 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: Aug 02 10:51:04.797849 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73e04b1-cb85-4185-ade1-dd0f4f862b32 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:51:04.914338 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c73e04b1-cb85-4185-ade1-dd0f4f862b32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2dee76b1e864cff867db2ae405abfe3', 'request_id': 'req-c73e04b1-cb85-4185-ade1-dd0f4f862b32'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:51:04.928425 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c73e04b1-cb85-4185-ade1-dd0f4f862b32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2dee76b1e864cff867db2ae405abfe3', 'request_id': 'req-c73e04b1-cb85-4185-ade1-dd0f4f862b32'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:51:05.004934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c73e04b1-cb85-4185-ade1-dd0f4f862b32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2dee76b1e864cff867db2ae405abfe3', 'request_id': 'req-c73e04b1-cb85-4185-ade1-dd0f4f862b32'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:51:05.031261 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c73e04b1-cb85-4185-ade1-dd0f4f862b32 admin admin] The token cache is being invalidate because project b2dee76b1e864cff867db2ae405abfe3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:51:05.031551 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c73e04b1-cb85-4185-ade1-dd0f4f862b32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:51:05.031983 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c73e04b1-cb85-4185-ade1-dd0f4f862b32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:51:05.033467 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 761/1510] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:51:04 2022] DELETE /identity/v3/projects/b2dee76b1e864cff867db2ae405abfe3 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:51:14.330292 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=62 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:51:15.029104 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:52:48.423775 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73145a7d-0533-42b9-95a7-58b6d1e3b6ed None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:52:48.424436 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73145a7d-0533-42b9-95a7-58b6d1e3b6ed None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:52:48.424436 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73145a7d-0533-42b9-95a7-58b6d1e3b6ed None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:52:48.425680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 750/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:52:48 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:52:48.430797 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4d85c-9ac7-45c7-9ba7-7418039fe872 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:52:48.431295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4d85c-9ac7-45c7-9ba7-7418039fe872 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:52:48.431394 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4d85c-9ac7-45c7-9ba7-7418039fe872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:52:48.515361 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-36e4d85c-9ac7-45c7-9ba7-7418039fe872 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:52:48.516605 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-36e4d85c-9ac7-45c7-9ba7-7418039fe872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:52:48.624845 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 762/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:52:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:52:48.782071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73145a7d-0533-42b9-95a7-58b6d1e3b6ed None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:52:48.784170 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-73145a7d-0533-42b9-95a7-58b6d1e3b6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:52:48.903001 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73145a7d-0533-42b9-95a7-58b6d1e3b6ed None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:52:48.908422 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6239257e-789a-40bc-a610-2b78e17ed833 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:52:48.909030 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239257e-789a-40bc-a610-2b78e17ed833 service nova] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:52:48.909303 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239257e-789a-40bc-a610-2b78e17ed833 service nova] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:52:48.909575 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6239257e-789a-40bc-a610-2b78e17ed833 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:52:48.917845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239257e-789a-40bc-a610-2b78e17ed833 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:52:48.919710 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6239257e-789a-40bc-a610-2b78e17ed833 service nova] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:52:48.929687 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 751/1513] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:52:48 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:52:49.791990 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904774a-fb74-4685-a4a7-c205ab3aee51 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:52:49.792562 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904774a-fb74-4685-a4a7-c205ab3aee51 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:52:49.792562 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f904774a-fb74-4685-a4a7-c205ab3aee51 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:52:49.793972 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 763/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:52:49 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:52:49.799323 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f7cd5-0b02-4875-97e3-4488ee45e0a5 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:52:49.799753 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f7cd5-0b02-4875-97e3-4488ee45e0a5 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:52:49.799861 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f7cd5-0b02-4875-97e3-4488ee45e0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:52:49.820029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-883f7cd5-0b02-4875-97e3-4488ee45e0a5 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:52:49.821211 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-883f7cd5-0b02-4875-97e3-4488ee45e0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:52:49.829000 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 752/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:52:49 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:52:49.983840 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904774a-fb74-4685-a4a7-c205ab3aee51 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:52:49.998667 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f904774a-fb74-4685-a4a7-c205ab3aee51 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:52:50.004133 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636a4cf7-e167-4e75-a91d-6bb9430588d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:52:50.004743 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a4cf7-e167-4e75-a91d-6bb9430588d0 service nova] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:52:50.005028 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a4cf7-e167-4e75-a91d-6bb9430588d0 service nova] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:52:50.005522 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636a4cf7-e167-4e75-a91d-6bb9430588d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:52:50.015581 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a4cf7-e167-4e75-a91d-6bb9430588d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:52:50.017238 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636a4cf7-e167-4e75-a91d-6bb9430588d0 service nova] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:52:50.028055 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 764/1516] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:52:49 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:52:51.118786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faac101-baa7-4548-b114-89aa7c203030 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:52:51.119458 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faac101-baa7-4548-b114-89aa7c203030 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:52:51.119990 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faac101-baa7-4548-b114-89aa7c203030 None None] PATH_INFO: `/v3` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:52:51.122380 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 753/1517] 10.222.0.107 () {54 vars in 930 bytes} [Tue Aug 2 10:52:51 2022] GET /identity/v3 => generated 256 bytes in 6 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:52:51.129210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cfa79-8b22-4aed-a1bc-23b58f8495d1 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:52:51.129681 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cfa79-8b22-4aed-a1bc-23b58f8495d1 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:52:51.129797 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4cfa79-8b22-4aed-a1bc-23b58f8495d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:52:51.202695 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-3d4cfa79-8b22-4aed-a1bc-23b58f8495d1 None None] MFA Rules not processed for user `cbc4b962e2a742b08beef7187cc527e5`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:52:51.203964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-3d4cfa79-8b22-4aed-a1bc-23b58f8495d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:52:51.221592 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:52:51.283875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 765/1518] 10.222.0.107 () {58 vars in 1020 bytes} [Tue Aug 2 10:52:51 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:52:55.304005 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06573650-7ecb-45a8-b9c7-490666ed75be None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:52:55.304597 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06573650-7ecb-45a8-b9c7-490666ed75be None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:52:55.304597 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06573650-7ecb-45a8-b9c7-490666ed75be None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:52:55.305932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 754/1519] 10.222.0.115 () {54 vars in 966 bytes} [Tue Aug 2 10:52:55 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:52:55.311166 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71a2bc-768e-46b1-bd5b-914f5d9c1946 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:52:55.311637 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71a2bc-768e-46b1-bd5b-914f5d9c1946 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:52:55.311751 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71a2bc-768e-46b1-bd5b-914f5d9c1946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:52:55.331737 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-1b71a2bc-768e-46b1-bd5b-914f5d9c1946 None None] MFA Rules not processed for user `cbc4b962e2a742b08beef7187cc527e5`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:52:55.332942 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-1b71a2bc-768e-46b1-bd5b-914f5d9c1946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:52:55.342179 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 766/1520] 10.222.0.115 () {58 vars in 1063 bytes} [Tue Aug 2 10:52:55 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:52:55.350420 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06573650-7ecb-45a8-b9c7-490666ed75be None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:52:55.363682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06573650-7ecb-45a8-b9c7-490666ed75be None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:52:55.369384 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc7399-014a-4999-845d-6d307e6f41ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc4b962e2a742b08beef7187cc527e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:52:55.369986 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc7399-014a-4999-845d-6d307e6f41ba service placement] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:52:55.370269 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc7399-014a-4999-845d-6d307e6f41ba service placement] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:52:55.370542 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc7399-014a-4999-845d-6d307e6f41ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:52:55.378977 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc7399-014a-4999-845d-6d307e6f41ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:52:55.380601 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc7399-014a-4999-845d-6d307e6f41ba service placement] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:52:55.390556 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 755/1521] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:52:55 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:53:01.279362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8addb83-06a8-4216-9d37-c207f7d86d09 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:53:01.279923 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8addb83-06a8-4216-9d37-c207f7d86d09 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:53:01.279923 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8addb83-06a8-4216-9d37-c207f7d86d09 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:53:01.281269 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 767/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:53:01 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:53:01.286056 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569251a-414c-40f3-87dd-419dcdbddd38 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:53:01.286521 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569251a-414c-40f3-87dd-419dcdbddd38 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:53:01.286642 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569251a-414c-40f3-87dd-419dcdbddd38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:53:01.306920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-d569251a-414c-40f3-87dd-419dcdbddd38 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:53:01.308151 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-d569251a-414c-40f3-87dd-419dcdbddd38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:53:01.316001 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 756/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:53:01 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:53:01.473853 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8addb83-06a8-4216-9d37-c207f7d86d09 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:53:01.488705 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8addb83-06a8-4216-9d37-c207f7d86d09 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:53:01.495184 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54086907-f1c3-40d6-81ac-048b4ea96fa7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:53:01.495793 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54086907-f1c3-40d6-81ac-048b4ea96fa7 service nova] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:53:01.496071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54086907-f1c3-40d6-81ac-048b4ea96fa7 service nova] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:53:01.496341 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54086907-f1c3-40d6-81ac-048b4ea96fa7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:53:01.506316 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54086907-f1c3-40d6-81ac-048b4ea96fa7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:53:01.511588 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54086907-f1c3-40d6-81ac-048b4ea96fa7 service nova] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:53:01.517958 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 768/1524] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:53:01 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:53:02.350680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba201c-c248-40c0-96ff-7d2b09e4d9f9 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:53:02.351559 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba201c-c248-40c0-96ff-7d2b09e4d9f9 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:53:02.351559 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dba201c-c248-40c0-96ff-7d2b09e4d9f9 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:53:02.353721 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 757/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 2 10:53:02 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:53:02.358711 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db415eb5-422b-4947-afc0-700df091ae4c None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:53:02.359332 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db415eb5-422b-4947-afc0-700df091ae4c None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:53:02.359332 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db415eb5-422b-4947-afc0-700df091ae4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:53:02.381822 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-db415eb5-422b-4947-afc0-700df091ae4c None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:53:02.383042 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-db415eb5-422b-4947-afc0-700df091ae4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:53:02.393293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 769/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 2 10:53:02 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:53:02.574765 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba201c-c248-40c0-96ff-7d2b09e4d9f9 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:53:02.592203 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dba201c-c248-40c0-96ff-7d2b09e4d9f9 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:53:02.598427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c1823c-70b3-4332-ae40-538c4ce1d890 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:53:02.599574 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1823c-70b3-4332-ae40-538c4ce1d890 service nova] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:53:02.599857 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1823c-70b3-4332-ae40-538c4ce1d890 service nova] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:53:02.600126 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c1823c-70b3-4332-ae40-538c4ce1d890 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:53:02.609391 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c1823c-70b3-4332-ae40-538c4ce1d890 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:53:02.610919 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c1823c-70b3-4332-ae40-538c4ce1d890 service nova] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:53:02.622524 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 758/1527] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:53:02 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:53:06.507497 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f86976-3461-4507-aed8-36a33cbcbae5 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:53:06.508076 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f86976-3461-4507-aed8-36a33cbcbae5 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:53:06.508181 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f86976-3461-4507-aed8-36a33cbcbae5 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:53:06.509475 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 770/1528] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:53:06 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:53:06.515074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55da5f1-5436-40c6-9ac0-75af9fd4f0d1 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:53:06.515598 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55da5f1-5436-40c6-9ac0-75af9fd4f0d1 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:53:06.515724 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55da5f1-5436-40c6-9ac0-75af9fd4f0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:53:06.535467 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-f55da5f1-5436-40c6-9ac0-75af9fd4f0d1 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:53:06.537233 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f55da5f1-5436-40c6-9ac0-75af9fd4f0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:53:06.545554 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 759/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:53:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:53:06.623940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f86976-3461-4507-aed8-36a33cbcbae5 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:53:06.625755 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-16f86976-3461-4507-aed8-36a33cbcbae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:53:06.709327 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f86976-3461-4507-aed8-36a33cbcbae5 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:53:06.712994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b763baf-c84f-4d0a-8f47-f19365babb93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:53:06.713658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b763baf-c84f-4d0a-8f47-f19365babb93 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:53:06.713941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b763baf-c84f-4d0a-8f47-f19365babb93 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:53:06.714217 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b763baf-c84f-4d0a-8f47-f19365babb93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:53:06.723368 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b763baf-c84f-4d0a-8f47-f19365babb93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:53:06.725649 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b763baf-c84f-4d0a-8f47-f19365babb93 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:53:06.735396 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 771/1530] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:53:06 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:53:08.762440 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4de94-3d55-417c-af08-2861a0c9ee2f None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:53:08.763161 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4de94-3d55-417c-af08-2861a0c9ee2f None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:53:08.763161 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4de94-3d55-417c-af08-2861a0c9ee2f None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:53:08.764378 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 760/1531] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:53:08 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:53:08.769628 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d1775-9855-4ad6-badf-1194bc673f53 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:53:08.770060 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d1775-9855-4ad6-badf-1194bc673f53 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:53:08.770171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d1775-9855-4ad6-badf-1194bc673f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:53:08.792716 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-710d1775-9855-4ad6-badf-1194bc673f53 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:53:08.794228 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-710d1775-9855-4ad6-badf-1194bc673f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:53:08.802873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 772/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:53:08 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:53:08.839453 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4de94-3d55-417c-af08-2861a0c9ee2f None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:53:08.841134 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-00f4de94-3d55-417c-af08-2861a0c9ee2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:53:08.854584 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4de94-3d55-417c-af08-2861a0c9ee2f None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:53:08.858040 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd392fc-6eb9-446e-871b-7b619cb85629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:53:08.858639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd392fc-6eb9-446e-871b-7b619cb85629 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:53:08.858919 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd392fc-6eb9-446e-871b-7b619cb85629 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:53:08.859229 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd392fc-6eb9-446e-871b-7b619cb85629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:53:08.869013 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd392fc-6eb9-446e-871b-7b619cb85629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:53:08.870437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd392fc-6eb9-446e-871b-7b619cb85629 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:53:08.879658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 761/1533] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:53:08 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:53:12.696358 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdcb54f-b7c6-481f-9278-b8f3f9c253a0 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:53:12.696935 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdcb54f-b7c6-481f-9278-b8f3f9c253a0 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:53:12.696935 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdcb54f-b7c6-481f-9278-b8f3f9c253a0 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:53:12.698245 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 773/1534] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:53:12 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:53:12.705676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbdb1c-b50b-428c-a1d0-1107b240c383 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:53:12.706121 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbdb1c-b50b-428c-a1d0-1107b240c383 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:53:12.706241 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fbdb1c-b50b-428c-a1d0-1107b240c383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:53:12.727217 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-e1fbdb1c-b50b-428c-a1d0-1107b240c383 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:53:12.728361 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-e1fbdb1c-b50b-428c-a1d0-1107b240c383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:53:12.736235 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 762/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:53:12 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:53:13.049011 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdcb54f-b7c6-481f-9278-b8f3f9c253a0 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:53:13.050720 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-1bdcb54f-b7c6-481f-9278-b8f3f9c253a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:53:13.118143 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:53:13.134534 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdcb54f-b7c6-481f-9278-b8f3f9c253a0 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:53:13.138228 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08580b5c-5ab3-4500-bcf9-f1f3b64706db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:53:13.138866 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08580b5c-5ab3-4500-bcf9-f1f3b64706db service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:53:13.139186 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08580b5c-5ab3-4500-bcf9-f1f3b64706db service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:53:13.139467 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08580b5c-5ab3-4500-bcf9-f1f3b64706db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:53:13.149772 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08580b5c-5ab3-4500-bcf9-f1f3b64706db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:53:13.151346 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08580b5c-5ab3-4500-bcf9-f1f3b64706db service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:53:13.161024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 774/1536] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:53:13 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:53:15.225616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567c8ee-18ff-496a-ba89-863b153e55a2 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:53:15.226238 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567c8ee-18ff-496a-ba89-863b153e55a2 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:53:15.226238 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567c8ee-18ff-496a-ba89-863b153e55a2 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:53:15.227567 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 763/1537] 127.0.0.1 () {54 vars in 916 bytes} [Tue Aug 2 10:53:15 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:53:15.232842 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31d728-ebf0-430d-be72-d7d5e782c928 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:53:15.233299 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31d728-ebf0-430d-be72-d7d5e782c928 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:53:15.233474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d31d728-ebf0-430d-be72-d7d5e782c928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:53:15.253170 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-4d31d728-ebf0-430d-be72-d7d5e782c928 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:53:15.254412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-4d31d728-ebf0-430d-be72-d7d5e782c928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:53:15.263159 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 775/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Tue Aug 2 10:53:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:53:15.573629 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567c8ee-18ff-496a-ba89-863b153e55a2 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:53:15.575371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-1567c8ee-18ff-496a-ba89-863b153e55a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:53:15.583606 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=84 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:53:15.591142 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567c8ee-18ff-496a-ba89-863b153e55a2 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:53:15.593327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84c30c-1786-45ba-9760-2a9226e5fb26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:53:15.593963 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84c30c-1786-45ba-9760-2a9226e5fb26 service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:53:15.594248 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84c30c-1786-45ba-9760-2a9226e5fb26 service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:53:15.594516 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84c30c-1786-45ba-9760-2a9226e5fb26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:53:15.604080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc84c30c-1786-45ba-9760-2a9226e5fb26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:53:15.605611 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc84c30c-1786-45ba-9760-2a9226e5fb26 service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:53:15.615439 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 764/1539] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:53:15 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:53:25.248515 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=11 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:53:25.662868 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=5 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:54:47.200583 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17116b-5762-4316-bbe4-0dc3a07c9b14 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:47.201772 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17116b-5762-4316-bbe4-0dc3a07c9b14 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:47.202278 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17116b-5762-4316-bbe4-0dc3a07c9b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:47.231805 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-1d17116b-5762-4316-bbe4-0dc3a07c9b14 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:47.233017 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-1d17116b-5762-4316-bbe4-0dc3a07c9b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:47.244379 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 776/1540] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:47.254645 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a665389-ea29-4750-af13-5dee721c26d0 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:47.257174 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a665389-ea29-4750-af13-5dee721c26d0 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:47.257174 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a665389-ea29-4750-af13-5dee721c26d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:47.280613 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92c1b1-77c1-4b7a-814b-35ea15c8d167 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:47.281059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92c1b1-77c1-4b7a-814b-35ea15c8d167 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:47.281250 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92c1b1-77c1-4b7a-814b-35ea15c8d167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:47.285807 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-5a665389-ea29-4750-af13-5dee721c26d0 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:47.287365 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-5a665389-ea29-4750-af13-5dee721c26d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:47.313727 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-4b92c1b1-77c1-4b7a-814b-35ea15c8d167 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:47.316912 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 765/1541] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:47.318220 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-4b92c1b1-77c1-4b7a-814b-35ea15c8d167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:47.324788 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a665389-ea29-4750-af13-5dee721c26d0 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:47.331825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 777/1542] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:54:47.333144 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87dff94-3646-4974-bb07-e62e65593935 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:47.333410 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87dff94-3646-4974-bb07-e62e65593935 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:47.333680 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87dff94-3646-4974-bb07-e62e65593935 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:47.337984 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a665389-ea29-4750-af13-5dee721c26d0 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:47.356855 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea28cd19-17c1-422f-b67a-a5a1a8f711fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:47.357833 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28cd19-17c1-422f-b67a-a5a1a8f711fe admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:47.358316 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28cd19-17c1-422f-b67a-a5a1a8f711fe admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:47.358793 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea28cd19-17c1-422f-b67a-a5a1a8f711fe admin admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:47.359754 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28cd19-17c1-422f-b67a-a5a1a8f711fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:54:47.360334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28cd19-17c1-422f-b67a-a5a1a8f711fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:47.361583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: WARNING py.warnings [None req-ea28cd19-17c1-422f-b67a-a5a1a8f711fe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:54:47.361583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: warnings.warn(msg) Aug 02 10:54:47.361583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: Aug 02 10:54:47.362856 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea28cd19-17c1-422f-b67a-a5a1a8f711fe admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:47.371524 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-c87dff94-3646-4974-bb07-e62e65593935 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:47.372732 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-c87dff94-3646-4974-bb07-e62e65593935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:47.382371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 766/1543] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:54:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:54:47.388448 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3e3d4-8373-4578-9ddc-ad3ebb95cbce None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:47.388982 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3e3d4-8373-4578-9ddc-ad3ebb95cbce None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:47.389365 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 778/1544] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:54:47.389663 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3e3d4-8373-4578-9ddc-ad3ebb95cbce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:47.398590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab3a86-80bb-4e06-974c-6c7579037739 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:47.398892 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab3a86-80bb-4e06-974c-6c7579037739 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:47.399197 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fab3a86-80bb-4e06-974c-6c7579037739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:47.424077 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-f8e3e3d4-8373-4578-9ddc-ad3ebb95cbce None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:47.425692 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f8e3e3d4-8373-4578-9ddc-ad3ebb95cbce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:47.431104 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-1fab3a86-80bb-4e06-974c-6c7579037739 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:47.432405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-1fab3a86-80bb-4e06-974c-6c7579037739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:47.440541 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 767/1545] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:47.441666 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3e3d4-8373-4578-9ddc-ad3ebb95cbce None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:47.450214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 779/1546] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:54:47.451090 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab3a86-80bb-4e06-974c-6c7579037739 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:47.462290 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3e3d4-8373-4578-9ddc-ad3ebb95cbce None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:47.472419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fab3a86-80bb-4e06-974c-6c7579037739 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:47.478013 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f6267-4015-4218-a837-c47417ef4b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:47.478644 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f6267-4015-4218-a837-c47417ef4b77 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:47.478985 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f6267-4015-4218-a837-c47417ef4b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:47.479223 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273f6267-4015-4218-a837-c47417ef4b77 admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:47.479927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f6267-4015-4218-a837-c47417ef4b77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:54:47.480239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f6267-4015-4218-a837-c47417ef4b77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:47.481877 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46790d-d7e6-4a98-971d-e0bfcfa2efbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:47.482031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273f6267-4015-4218-a837-c47417ef4b77 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:47.482884 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46790d-d7e6-4a98-971d-e0bfcfa2efbb admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:47.483709 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46790d-d7e6-4a98-971d-e0bfcfa2efbb admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:47.484229 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be46790d-d7e6-4a98-971d-e0bfcfa2efbb admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:47.485137 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be46790d-d7e6-4a98-971d-e0bfcfa2efbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:47.487321 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be46790d-d7e6-4a98-971d-e0bfcfa2efbb admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:47.523759 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 780/1547] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:54:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:54:47.525264 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f6267-4015-4218-a837-c47417ef4b77 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:47.559512 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273f6267-4015-4218-a837-c47417ef4b77 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:47.573337 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fda022-2166-40ee-ad1f-426d8c9da642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:47.576154 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fda022-2166-40ee-ad1f-426d8c9da642 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:47.576154 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fda022-2166-40ee-ad1f-426d8c9da642 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:47.576154 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fda022-2166-40ee-ad1f-426d8c9da642 admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:47.576908 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fda022-2166-40ee-ad1f-426d8c9da642 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:54:47.577468 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fda022-2166-40ee-ad1f-426d8c9da642 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:47.585439 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fda022-2166-40ee-ad1f-426d8c9da642 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:47.616783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 768/1548] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:54:47 2022] POST /identity/v3/projects => generated 380 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:54:47.622803 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 781/1549] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:54:47 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:54:47.622803 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d58d01-33a2-4549-93ca-3dc16791c37f None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:47.623149 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d58d01-33a2-4549-93ca-3dc16791c37f None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:47.623365 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d58d01-33a2-4549-93ca-3dc16791c37f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:47.624283 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46790d-d7e6-4a98-971d-e0bfcfa2efbb admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:47.690868 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-48d58d01-33a2-4549-93ca-3dc16791c37f None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:47.691840 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-48d58d01-33a2-4549-93ca-3dc16791c37f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:47.738009 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be46790d-d7e6-4a98-971d-e0bfcfa2efbb admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:47.744434 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b93366-fe72-4bf2-8a45-3ae513acd763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:47.745054 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b93366-fe72-4bf2-8a45-3ae513acd763 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:47.745336 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b93366-fe72-4bf2-8a45-3ae513acd763 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:47.745610 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b93366-fe72-4bf2-8a45-3ae513acd763 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:47.746287 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b93366-fe72-4bf2-8a45-3ae513acd763 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:47.748249 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b93366-fe72-4bf2-8a45-3ae513acd763 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:47.771120 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 782/1550] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:54:47.771950 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d58d01-33a2-4549-93ca-3dc16791c37f None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:47.781775 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d58d01-33a2-4549-93ca-3dc16791c37f None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:47.786525 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0af262-3221-42db-92af-af3bf58963c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:47.787166 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0af262-3221-42db-92af-af3bf58963c9 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:47.787457 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0af262-3221-42db-92af-af3bf58963c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:47.787737 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0af262-3221-42db-92af-af3bf58963c9 admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:47.788484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0af262-3221-42db-92af-af3bf58963c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:47.790672 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0af262-3221-42db-92af-af3bf58963c9 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:47.827411 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 769/1551] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:54:47 2022] POST /identity/v3/projects => generated 404 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:54:47.828712 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b93366-fe72-4bf2-8a45-3ae513acd763 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:47.888946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b93366-fe72-4bf2-8a45-3ae513acd763 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:47.894388 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de4400-903d-4c38-b1a3-1053cf67b493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:47.894997 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de4400-903d-4c38-b1a3-1053cf67b493 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:47.895322 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de4400-903d-4c38-b1a3-1053cf67b493 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:47.895594 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de4400-903d-4c38-b1a3-1053cf67b493 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:47.896252 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de4400-903d-4c38-b1a3-1053cf67b493 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:47.898147 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20de4400-903d-4c38-b1a3-1053cf67b493 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:47.900206 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 783/1552] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:54:47 2022] POST /identity/v3/users => generated 291 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:54:47.901556 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed107f-ff24-47f6-8088-d7f0ae673620 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:47.901806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed107f-ff24-47f6-8088-d7f0ae673620 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:47.902082 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ed107f-ff24-47f6-8088-d7f0ae673620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:47.923820 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-60ed107f-ff24-47f6-8088-d7f0ae673620 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:47.925016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-60ed107f-ff24-47f6-8088-d7f0ae673620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:47.932899 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 784/1553] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:47.934256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43068a09-c053-4011-a18c-efc9cdefdbed None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:47.934539 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43068a09-c053-4011-a18c-efc9cdefdbed None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:47.934816 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43068a09-c053-4011-a18c-efc9cdefdbed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:47.957928 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-43068a09-c053-4011-a18c-efc9cdefdbed None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:47.959890 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-43068a09-c053-4011-a18c-efc9cdefdbed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:47.967779 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 785/1554] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:47.969477 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a286dde-fcd5-45e1-bfb6-faec6bbae6be None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:47.969796 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a286dde-fcd5-45e1-bfb6-faec6bbae6be None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:47.970073 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a286dde-fcd5-45e1-bfb6-faec6bbae6be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:47.980546 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 770/1555] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:54:47 2022] POST /identity/v3/projects => generated 388 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:54:47.981802 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd75fc4-490e-4653-a332-04e16bfc6919 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:47.982088 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd75fc4-490e-4653-a332-04e16bfc6919 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:47.982365 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd75fc4-490e-4653-a332-04e16bfc6919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:47.990834 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-5a286dde-fcd5-45e1-bfb6-faec6bbae6be None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:47.992059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-5a286dde-fcd5-45e1-bfb6-faec6bbae6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:48.003726 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-dcd75fc4-490e-4653-a332-04e16bfc6919 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:48.004926 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-dcd75fc4-490e-4653-a332-04e16bfc6919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:48.013971 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:54:48.064090 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 786/1556] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:48.064090 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bb5e4-bfad-40ab-813e-48d8716aeb2a None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:48.064090 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bb5e4-bfad-40ab-813e-48d8716aeb2a None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:48.064090 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15bb5e4-bfad-40ab-813e-48d8716aeb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:48.068772 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 771/1557] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:48.069596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd75fc4-490e-4653-a332-04e16bfc6919 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:48.079704 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd75fc4-490e-4653-a332-04e16bfc6919 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:48.083262 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-f15bb5e4-bfad-40ab-813e-48d8716aeb2a None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:48.084366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc2b01-bc43-4f1b-a941-0e625e059f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:48.084517 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-f15bb5e4-bfad-40ab-813e-48d8716aeb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:48.084977 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc2b01-bc43-4f1b-a941-0e625e059f4a admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:48.085251 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc2b01-bc43-4f1b-a941-0e625e059f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:48.085516 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bc2b01-bc43-4f1b-a941-0e625e059f4a admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:48.086170 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc2b01-bc43-4f1b-a941-0e625e059f4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:48.087314 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: WARNING py.warnings [None req-b5bc2b01-bc43-4f1b-a941-0e625e059f4a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:54:48.087314 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: warnings.warn(msg) Aug 02 10:54:48.087314 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: Aug 02 10:54:48.088860 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bc2b01-bc43-4f1b-a941-0e625e059f4a admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:48.092751 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 787/1558] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:48.093653 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15bb5e4-bfad-40ab-813e-48d8716aeb2a None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:48.106065 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15bb5e4-bfad-40ab-813e-48d8716aeb2a None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:48.110891 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582002aa-119c-4c44-a5df-26621e720603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:48.114802 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582002aa-119c-4c44-a5df-26621e720603 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:48.114802 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582002aa-119c-4c44-a5df-26621e720603 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:48.114802 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582002aa-119c-4c44-a5df-26621e720603 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:48.114802 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582002aa-119c-4c44-a5df-26621e720603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:48.114802 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582002aa-119c-4c44-a5df-26621e720603 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:48.120008 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:54:48.128439 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 788/1559] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:54:48 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:54:48.128439 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da5835d-f68f-4c14-84b1-eb342d1ac43a None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:48.128439 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da5835d-f68f-4c14-84b1-eb342d1ac43a None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:48.128439 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da5835d-f68f-4c14-84b1-eb342d1ac43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:48.147281 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-0da5835d-f68f-4c14-84b1-eb342d1ac43a None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:48.148456 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-0da5835d-f68f-4c14-84b1-eb342d1ac43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:48.156627 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 789/1560] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:48.157782 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e1298-4609-44d4-a47e-7a33786e5921 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:48.158062 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e1298-4609-44d4-a47e-7a33786e5921 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:48.158336 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e1298-4609-44d4-a47e-7a33786e5921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:48.178307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-511e1298-4609-44d4-a47e-7a33786e5921 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:48.183003 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-511e1298-4609-44d4-a47e-7a33786e5921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:48.183184 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 772/1561] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:54:48 2022] POST /identity/v3/users => generated 303 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:54:48.183184 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc2b01-bc43-4f1b-a941-0e625e059f4a admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:48.189890 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 790/1562] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:48.192342 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bc2b01-bc43-4f1b-a941-0e625e059f4a admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:48.194756 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97ead6-6141-4242-8b7a-bd5e7e916c5c None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:48.195069 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97ead6-6141-4242-8b7a-bd5e7e916c5c None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:48.195429 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97ead6-6141-4242-8b7a-bd5e7e916c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:48.197815 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ccce8-3386-4c04-a5db-1f41c397144e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:48.198428 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ccce8-3386-4c04-a5db-1f41c397144e admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:48.198707 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ccce8-3386-4c04-a5db-1f41c397144e admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:48.198981 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ccce8-3386-4c04-a5db-1f41c397144e admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:48.199893 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ccce8-3386-4c04-a5db-1f41c397144e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:48.201820 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ccce8-3386-4c04-a5db-1f41c397144e admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:48.217240 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-8d97ead6-6141-4242-8b7a-bd5e7e916c5c None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:48.223122 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-8d97ead6-6141-4242-8b7a-bd5e7e916c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:48.226556 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 791/1563] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:48.227917 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2fec5b-e7cb-49c1-a1db-7611186f2cb6 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:48.228203 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2fec5b-e7cb-49c1-a1db-7611186f2cb6 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:48.228478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2fec5b-e7cb-49c1-a1db-7611186f2cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:48.248535 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-4e2fec5b-e7cb-49c1-a1db-7611186f2cb6 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:48.251812 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-4e2fec5b-e7cb-49c1-a1db-7611186f2cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:48.258728 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 792/1564] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:48.259626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2fec5b-e7cb-49c1-a1db-7611186f2cb6 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:48.270090 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2fec5b-e7cb-49c1-a1db-7611186f2cb6 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:48.275566 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7341fbf-464b-4035-a4ad-45300bc17d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:48.275566 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7341fbf-464b-4035-a4ad-45300bc17d10 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:48.275774 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7341fbf-464b-4035-a4ad-45300bc17d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:48.275991 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7341fbf-464b-4035-a4ad-45300bc17d10 admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:48.276681 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7341fbf-464b-4035-a4ad-45300bc17d10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:54:48.276987 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7341fbf-464b-4035-a4ad-45300bc17d10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:48.277671 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 773/1565] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:54:48 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:54:48.278722 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7341fbf-464b-4035-a4ad-45300bc17d10 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:48.281355 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ccce8-3386-4c04-a5db-1f41c397144e admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:48.291784 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ccce8-3386-4c04-a5db-1f41c397144e admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:48.291972 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 793/1566] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:54:48 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:54:48.292844 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7341fbf-464b-4035-a4ad-45300bc17d10 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:48.303702 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7341fbf-464b-4035-a4ad-45300bc17d10 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:48.304278 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41461d2b-adff-48a0-97ed-4d4ae8866c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:48.309158 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41461d2b-adff-48a0-97ed-4d4ae8866c23 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:48.309158 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41461d2b-adff-48a0-97ed-4d4ae8866c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:48.309158 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41461d2b-adff-48a0-97ed-4d4ae8866c23 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:48.309158 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41461d2b-adff-48a0-97ed-4d4ae8866c23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:48.309158 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41461d2b-adff-48a0-97ed-4d4ae8866c23 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:48.309564 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f325b75-2bc8-44f6-9df8-6715f2d2e467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:48.309564 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f325b75-2bc8-44f6-9df8-6715f2d2e467 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:48.309564 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f325b75-2bc8-44f6-9df8-6715f2d2e467 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:48.309803 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f325b75-2bc8-44f6-9df8-6715f2d2e467 admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:48.310563 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f325b75-2bc8-44f6-9df8-6715f2d2e467 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:54:48.310891 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f325b75-2bc8-44f6-9df8-6715f2d2e467 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:48.313346 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f325b75-2bc8-44f6-9df8-6715f2d2e467 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:48.319767 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 774/1567] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:54:48 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:54:48.320415 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41461d2b-adff-48a0-97ed-4d4ae8866c23 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:48.328310 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 794/1568] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:54:48 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:54:48.329263 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f325b75-2bc8-44f6-9df8-6715f2d2e467 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:48.339686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41461d2b-adff-48a0-97ed-4d4ae8866c23 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:48.340797 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f325b75-2bc8-44f6-9df8-6715f2d2e467 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:48.344496 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476e757-ce89-455f-8868-a8267ed841c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:48.345124 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476e757-ce89-455f-8868-a8267ed841c9 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:48.345411 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476e757-ce89-455f-8868-a8267ed841c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:48.345684 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476e757-ce89-455f-8868-a8267ed841c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:48.345905 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f181c4-2242-4505-b55f-cd0187d1dddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:48.346372 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476e757-ce89-455f-8868-a8267ed841c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:48.346512 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f181c4-2242-4505-b55f-cd0187d1dddf admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:48.346785 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f181c4-2242-4505-b55f-cd0187d1dddf admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:48.347077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f181c4-2242-4505-b55f-cd0187d1dddf admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:48.347772 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f181c4-2242-4505-b55f-cd0187d1dddf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:54:48.348028 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476e757-ce89-455f-8868-a8267ed841c9 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:48.348152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f181c4-2242-4505-b55f-cd0187d1dddf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:48.349639 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f181c4-2242-4505-b55f-cd0187d1dddf admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:48.359537 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 775/1569] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:54:48 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:54:48.363954 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476e757-ce89-455f-8868-a8267ed841c9 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:48.371911 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 795/1570] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:54:48 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:54:48.371911 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f181c4-2242-4505-b55f-cd0187d1dddf admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:48.376121 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f181c4-2242-4505-b55f-cd0187d1dddf admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:48.378604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476e757-ce89-455f-8868-a8267ed841c9 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:48.380788 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4134d-76ea-4774-a526-b0cbc9939a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:48.381396 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4134d-76ea-4774-a526-b0cbc9939a44 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:48.381679 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4134d-76ea-4774-a526-b0cbc9939a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:48.381991 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde4134d-76ea-4774-a526-b0cbc9939a44 admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:48.382642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde4134d-76ea-4774-a526-b0cbc9939a44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:54:48.383070 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde4134d-76ea-4774-a526-b0cbc9939a44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:48.383779 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963d692-9101-4272-a9aa-c396fd96cf3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:48.384393 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963d692-9101-4272-a9aa-c396fd96cf3d admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:48.384677 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde4134d-76ea-4774-a526-b0cbc9939a44 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:48.384999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963d692-9101-4272-a9aa-c396fd96cf3d admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:48.385282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a963d692-9101-4272-a9aa-c396fd96cf3d admin admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:48.386005 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a963d692-9101-4272-a9aa-c396fd96cf3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:54:48.386311 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a963d692-9101-4272-a9aa-c396fd96cf3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:48.388009 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a963d692-9101-4272-a9aa-c396fd96cf3d admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:48.398141 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 796/1571] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:54:48 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:54:48.399032 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4134d-76ea-4774-a526-b0cbc9939a44 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:48.408078 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 776/1572] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:54:48 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:54:48.408078 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963d692-9101-4272-a9aa-c396fd96cf3d admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:48.415770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a963d692-9101-4272-a9aa-c396fd96cf3d admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:48.416214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde4134d-76ea-4774-a526-b0cbc9939a44 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:48.420427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64521186-37c2-488c-936c-b2c365069358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:48.420768 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f556fe0-9e62-496d-911b-b602417a5f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:48.421036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64521186-37c2-488c-936c-b2c365069358 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:48.421316 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64521186-37c2-488c-936c-b2c365069358 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:48.421426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f556fe0-9e62-496d-911b-b602417a5f7a admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:48.422123 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64521186-37c2-488c-936c-b2c365069358 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:48.422335 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f556fe0-9e62-496d-911b-b602417a5f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:48.422622 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f556fe0-9e62-496d-911b-b602417a5f7a admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:48.423110 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64521186-37c2-488c-936c-b2c365069358 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:48.423454 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f556fe0-9e62-496d-911b-b602417a5f7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:48.425023 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f556fe0-9e62-496d-911b-b602417a5f7a admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:48.425129 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64521186-37c2-488c-936c-b2c365069358 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:48.435560 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 797/1573] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:54:48 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:54:48.436455 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f556fe0-9e62-496d-911b-b602417a5f7a admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:48.447556 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f556fe0-9e62-496d-911b-b602417a5f7a admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:48.452293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1e604-a61f-4153-9542-66cce782e9fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:48.455528 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1e604-a61f-4153-9542-66cce782e9fa admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:48.455528 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1e604-a61f-4153-9542-66cce782e9fa admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:48.455528 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a1e604-a61f-4153-9542-66cce782e9fa admin admin] PATH_INFO: `/v3/projects/5ced4a8a26d94c10b08bbca381d62d34/users/822e6ac5ec8347c9815c770738509d30/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:48.470801 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a1e604-a61f-4153-9542-66cce782e9fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ced4a8a26d94c10b08bbca381d62d34, user_id=822e6ac5ec8347c9815c770738509d30, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:48.471868 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: WARNING py.warnings [None req-80a1e604-a61f-4153-9542-66cce782e9fa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:54:48.471868 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: warnings.warn(msg) Aug 02 10:54:48.471868 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: Aug 02 10:54:48.473808 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a1e604-a61f-4153-9542-66cce782e9fa admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:48.507364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 777/1574] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:54:48 2022] POST /identity/v3/projects => generated 374 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:54:48.508314 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64521186-37c2-488c-936c-b2c365069358 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:48.523991 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 798/1575] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:54:48 2022] PUT /identity/v3/projects/5ced4a8a26d94c10b08bbca381d62d34/users/822e6ac5ec8347c9815c770738509d30/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:54:48.528013 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1e604-a61f-4153-9542-66cce782e9fa admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:48.571830 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64521186-37c2-488c-936c-b2c365069358 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:48.587407 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a1e604-a61f-4153-9542-66cce782e9fa admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:48.592256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2be697-3ce0-4a6b-bd96-5bf6ba9cc2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:48.592804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2be697-3ce0-4a6b-bd96-5bf6ba9cc2c4 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:48.593086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2be697-3ce0-4a6b-bd96-5bf6ba9cc2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:48.594028 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2be697-3ce0-4a6b-bd96-5bf6ba9cc2c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:48.595021 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2be697-3ce0-4a6b-bd96-5bf6ba9cc2c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:48.596648 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2be697-3ce0-4a6b-bd96-5bf6ba9cc2c4 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:48.598440 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:54:48.606622 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 799/1576] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:54:48 2022] GET /identity/v3/roles => generated 1427 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:54:48.610406 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2be697-3ce0-4a6b-bd96-5bf6ba9cc2c4 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:48.617100 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2be697-3ce0-4a6b-bd96-5bf6ba9cc2c4 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:48.621875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b70644-b793-485d-a1c5-ce9164bd179c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:48.622469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b70644-b793-485d-a1c5-ce9164bd179c admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:48.622751 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b70644-b793-485d-a1c5-ce9164bd179c admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:48.623028 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b70644-b793-485d-a1c5-ce9164bd179c admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:48.623716 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b70644-b793-485d-a1c5-ce9164bd179c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:48.624871 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: WARNING py.warnings [None req-79b70644-b793-485d-a1c5-ce9164bd179c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:54:48.624871 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: warnings.warn(msg) Aug 02 10:54:48.624871 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: Aug 02 10:54:48.626070 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b70644-b793-485d-a1c5-ce9164bd179c admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:48.633034 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1838b976-78ff-4e29-ad84-7bf05351d5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:48.633649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838b976-78ff-4e29-ad84-7bf05351d5f8 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:48.633919 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838b976-78ff-4e29-ad84-7bf05351d5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:48.634189 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1838b976-78ff-4e29-ad84-7bf05351d5f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:48.634843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1838b976-78ff-4e29-ad84-7bf05351d5f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:48.636764 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1838b976-78ff-4e29-ad84-7bf05351d5f8 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:48.747320 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 800/1577] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:54:48 2022] POST /identity/v3/projects => generated 386 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:54:48.748132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b70644-b793-485d-a1c5-ce9164bd179c admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:48.759186 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 778/1578] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:54:48 2022] POST /identity/v3/projects => generated 388 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:54:48.760198 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1838b976-78ff-4e29-ad84-7bf05351d5f8 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:48.770191 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:54:48.813479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b70644-b793-485d-a1c5-ce9164bd179c admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:48.825656 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1838b976-78ff-4e29-ad84-7bf05351d5f8 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:48.831098 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855ca59-4085-4d43-92a7-8ca4bb0b4e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:48.831715 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855ca59-4085-4d43-92a7-8ca4bb0b4e04 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:48.831999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855ca59-4085-4d43-92a7-8ca4bb0b4e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:48.832607 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855ca59-4085-4d43-92a7-8ca4bb0b4e04 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:48.832923 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855ca59-4085-4d43-92a7-8ca4bb0b4e04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:48.834859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855ca59-4085-4d43-92a7-8ca4bb0b4e04 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:48.870644 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b654c-c2de-4b73-bd81-a957d73decda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:48.871258 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b654c-c2de-4b73-bd81-a957d73decda admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:48.871546 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b654c-c2de-4b73-bd81-a957d73decda admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:48.871825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8b654c-c2de-4b73-bd81-a957d73decda admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:48.872490 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b654c-c2de-4b73-bd81-a957d73decda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:48.874434 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8b654c-c2de-4b73-bd81-a957d73decda admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:48.900094 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 779/1579] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:54:48 2022] POST /identity/v3/projects => generated 376 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:54:48.900094 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855ca59-4085-4d43-92a7-8ca4bb0b4e04 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:48.957809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855ca59-4085-4d43-92a7-8ca4bb0b4e04 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:48.962686 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 801/1580] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:54:48 2022] POST /identity/v3/projects => generated 376 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:54:48.962686 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b654c-c2de-4b73-bd81-a957d73decda admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:49.005271 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:54:49.019664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37855cca-8769-4a95-b176-798303c4a018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:49.020294 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37855cca-8769-4a95-b176-798303c4a018 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:49.020574 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37855cca-8769-4a95-b176-798303c4a018 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:49.020844 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37855cca-8769-4a95-b176-798303c4a018 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:49.021540 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37855cca-8769-4a95-b176-798303c4a018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:49.023167 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37855cca-8769-4a95-b176-798303c4a018 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:49.028305 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8b654c-c2de-4b73-bd81-a957d73decda admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:49.028804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c7b8d-4c0d-4269-9fe3-ecdbde04a5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:49.029426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c7b8d-4c0d-4269-9fe3-ecdbde04a5ed admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:49.029711 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c7b8d-4c0d-4269-9fe3-ecdbde04a5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:49.029989 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622c7b8d-4c0d-4269-9fe3-ecdbde04a5ed admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:49.030689 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c7b8d-4c0d-4269-9fe3-ecdbde04a5ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:49.032697 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622c7b8d-4c0d-4269-9fe3-ecdbde04a5ed admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:49.033897 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 780/1581] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:54:48 2022] GET /identity/v3/roles => generated 1427 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:54:49.034934 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5b4f7-46aa-445b-b91c-edcaaa83a39a None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:49.035427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5b4f7-46aa-445b-b91c-edcaaa83a39a None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:49.035710 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5b4f7-46aa-445b-b91c-edcaaa83a39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:49.056465 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-2aa5b4f7-46aa-445b-b91c-edcaaa83a39a None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:49.057812 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-2aa5b4f7-46aa-445b-b91c-edcaaa83a39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:49.066053 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 781/1582] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:49 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:49.066794 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5b4f7-46aa-445b-b91c-edcaaa83a39a None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:49.075963 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5b4f7-46aa-445b-b91c-edcaaa83a39a None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:49.081693 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfe1807-379b-4dd3-a34d-8ca227ac8696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:49.089372 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe1807-379b-4dd3-a34d-8ca227ac8696 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:49.089372 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe1807-379b-4dd3-a34d-8ca227ac8696 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:49.089372 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfe1807-379b-4dd3-a34d-8ca227ac8696 admin admin] PATH_INFO: `/v3/projects/e5e49c7e12b64966a1c1b92a50ba8347/users/b7fc4b12f4f541609ddf148c8b51d8f1/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:49.097441 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfe1807-379b-4dd3-a34d-8ca227ac8696 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5e49c7e12b64966a1c1b92a50ba8347, user_id=b7fc4b12f4f541609ddf148c8b51d8f1, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:49.100243 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfe1807-379b-4dd3-a34d-8ca227ac8696 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:49.127148 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 802/1583] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:54:48 2022] POST /identity/v3/users => generated 288 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:54:49.127148 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c7b8d-4c0d-4269-9fe3-ecdbde04a5ed admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:49.134641 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622c7b8d-4c0d-4269-9fe3-ecdbde04a5ed admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:49.139315 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4259fdd-6e9e-4d9c-bd3d-482f1648fdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:49.139918 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4259fdd-6e9e-4d9c-bd3d-482f1648fdde admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:49.140378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4259fdd-6e9e-4d9c-bd3d-482f1648fdde admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:49.140926 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4259fdd-6e9e-4d9c-bd3d-482f1648fdde admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:49.141253 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4259fdd-6e9e-4d9c-bd3d-482f1648fdde admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:49.143219 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4259fdd-6e9e-4d9c-bd3d-482f1648fdde admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:49.176871 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 782/1584] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:54:49 2022] PUT /identity/v3/projects/e5e49c7e12b64966a1c1b92a50ba8347/users/b7fc4b12f4f541609ddf148c8b51d8f1/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:54:49.177764 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfe1807-379b-4dd3-a34d-8ca227ac8696 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:49.219967 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 803/1585] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:54:49 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:54:49.221383 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4259fdd-6e9e-4d9c-bd3d-482f1648fdde admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:49.232255 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4259fdd-6e9e-4d9c-bd3d-482f1648fdde admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:49.237604 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912988d3-4bb7-4b76-864e-f9a8709a54a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:49.238217 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912988d3-4bb7-4b76-864e-f9a8709a54a3 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:49.238501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912988d3-4bb7-4b76-864e-f9a8709a54a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:49.238777 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912988d3-4bb7-4b76-864e-f9a8709a54a3 admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:49.239485 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912988d3-4bb7-4b76-864e-f9a8709a54a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:49.241395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912988d3-4bb7-4b76-864e-f9a8709a54a3 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:49.243171 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfe1807-379b-4dd3-a34d-8ca227ac8696 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:49.247775 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcff809-9e57-476b-b1b4-c283bec9007c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:49.248520 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcff809-9e57-476b-b1b4-c283bec9007c admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:49.248865 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcff809-9e57-476b-b1b4-c283bec9007c admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:49.249069 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcff809-9e57-476b-b1b4-c283bec9007c admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:49.250624 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcff809-9e57-476b-b1b4-c283bec9007c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:49.252898 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcff809-9e57-476b-b1b4-c283bec9007c admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:49.311296 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 804/1586] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:54:49 2022] POST /identity/v3/users => generated 289 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:54:49.312735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912988d3-4bb7-4b76-864e-f9a8709a54a3 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:49.322746 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912988d3-4bb7-4b76-864e-f9a8709a54a3 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:49.325962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 783/1587] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:54:49 2022] POST /identity/v3/users => generated 295 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:54:49.325962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcff809-9e57-476b-b1b4-c283bec9007c admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:49.327759 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c18e2-705b-45c9-ac76-5db011b21150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:49.328719 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c18e2-705b-45c9-ac76-5db011b21150 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:49.329070 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c18e2-705b-45c9-ac76-5db011b21150 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:49.329284 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c18e2-705b-45c9-ac76-5db011b21150 admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:49.329948 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c18e2-705b-45c9-ac76-5db011b21150 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:49.331948 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311c18e2-705b-45c9-ac76-5db011b21150 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:49.337811 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcff809-9e57-476b-b1b4-c283bec9007c admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:49.339127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd4723-098e-48b5-b89d-abc68f8416f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:49.339797 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd4723-098e-48b5-b89d-abc68f8416f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:49.340070 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:54:49.340192 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd4723-098e-48b5-b89d-abc68f8416f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:49.340329 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cd4723-098e-48b5-b89d-abc68f8416f3 admin admin] PATH_INFO: `/v3/projects/6d0f5621fd55478ca86c8fc95ddb48bf/users/93ddf7b48ffe441eb5379ef455c3dd9e/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:49.355867 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd4723-098e-48b5-b89d-abc68f8416f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d0f5621fd55478ca86c8fc95ddb48bf, user_id=93ddf7b48ffe441eb5379ef455c3dd9e, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:49.358525 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cd4723-098e-48b5-b89d-abc68f8416f3 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:49.423900 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 805/1588] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:54:49 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:54:49.423900 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c18e2-705b-45c9-ac76-5db011b21150 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:49.474029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 784/1589] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:54:49 2022] PUT /identity/v3/projects/6d0f5621fd55478ca86c8fc95ddb48bf/users/93ddf7b48ffe441eb5379ef455c3dd9e/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:54:49.474029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd4723-098e-48b5-b89d-abc68f8416f3 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:49.479697 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311c18e2-705b-45c9-ac76-5db011b21150 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:49.526741 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cd4723-098e-48b5-b89d-abc68f8416f3 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:49.531951 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9745e930-0fc9-4aae-a5f8-d042d63782ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:49.532200 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af83f8-2457-4115-a9d8-d2d18d9c3e0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:49.532567 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745e930-0fc9-4aae-a5f8-d042d63782ac admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:49.532843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745e930-0fc9-4aae-a5f8-d042d63782ac admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:49.532954 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af83f8-2457-4115-a9d8-d2d18d9c3e0c service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:49.533112 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9745e930-0fc9-4aae-a5f8-d042d63782ac admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:49.533332 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af83f8-2457-4115-a9d8-d2d18d9c3e0c service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:49.533674 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00af83f8-2457-4115-a9d8-d2d18d9c3e0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:49.533812 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745e930-0fc9-4aae-a5f8-d042d63782ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:49.535444 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9745e930-0fc9-4aae-a5f8-d042d63782ac admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:49.543488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af83f8-2457-4115-a9d8-d2d18d9c3e0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:49.544959 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00af83f8-2457-4115-a9d8-d2d18d9c3e0c service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:49.545879 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 785/1590] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:54:49 2022] GET /identity/v3/roles => generated 1427 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:54:49.550830 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1882bb7-2bba-4bca-b425-407f4ee076eb None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:49.551248 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1882bb7-2bba-4bca-b425-407f4ee076eb None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:49.551529 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1882bb7-2bba-4bca-b425-407f4ee076eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:49.555668 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 806/1591] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:54:49 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:54:49.556366 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af83f8-2457-4115-a9d8-d2d18d9c3e0c service neutron] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:49.566092 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00af83f8-2457-4115-a9d8-d2d18d9c3e0c service neutron] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:49.571081 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d31857-ed96-48a5-83cf-babd003724dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:49.571552 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d31857-ed96-48a5-83cf-babd003724dd admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:49.571758 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d31857-ed96-48a5-83cf-babd003724dd admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:49.572038 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d31857-ed96-48a5-83cf-babd003724dd admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:49.572657 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-b1882bb7-2bba-4bca-b425-407f4ee076eb None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:49.572796 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d31857-ed96-48a5-83cf-babd003724dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:49.574241 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-b1882bb7-2bba-4bca-b425-407f4ee076eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:49.574375 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d31857-ed96-48a5-83cf-babd003724dd admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:49.582957 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 786/1592] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:49 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:49.583851 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1882bb7-2bba-4bca-b425-407f4ee076eb None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:49.589884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 807/1593] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:54:49 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:54:49.589884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d31857-ed96-48a5-83cf-babd003724dd admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:49.594101 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1882bb7-2bba-4bca-b425-407f4ee076eb None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:49.599395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e39aff1-76c7-4a25-9459-1ee2cfd5c621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:49.600017 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39aff1-76c7-4a25-9459-1ee2cfd5c621 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:49.600297 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39aff1-76c7-4a25-9459-1ee2cfd5c621 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:49.600570 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39aff1-76c7-4a25-9459-1ee2cfd5c621 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:49.601265 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e39aff1-76c7-4a25-9459-1ee2cfd5c621 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:49.601512 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d31857-ed96-48a5-83cf-babd003724dd admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:49.602941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e39aff1-76c7-4a25-9459-1ee2cfd5c621 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:49.604414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:54:49.606417 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa385432-0e1e-4d70-8e86-2bd901647206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:49.607123 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa385432-0e1e-4d70-8e86-2bd901647206 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:49.607409 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa385432-0e1e-4d70-8e86-2bd901647206 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:49.607691 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa385432-0e1e-4d70-8e86-2bd901647206 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:49.608537 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa385432-0e1e-4d70-8e86-2bd901647206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:49.610867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa385432-0e1e-4d70-8e86-2bd901647206 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:49.617851 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 787/1594] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:54:49 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:54:49.617851 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e39aff1-76c7-4a25-9459-1ee2cfd5c621 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:49.621534 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 808/1595] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:54:49 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:54:49.623091 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1d9ee-ed3b-4c54-915a-603277d76694 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:49.623518 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1d9ee-ed3b-4c54-915a-603277d76694 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:49.623800 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a1d9ee-ed3b-4c54-915a-603277d76694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:49.627397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e39aff1-76c7-4a25-9459-1ee2cfd5c621 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:49.632247 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505e446-b823-40ac-b126-8f800eafb710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:49.632932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505e446-b823-40ac-b126-8f800eafb710 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:49.633331 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505e446-b823-40ac-b126-8f800eafb710 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:49.633610 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505e446-b823-40ac-b126-8f800eafb710 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:49.634305 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e505e446-b823-40ac-b126-8f800eafb710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:49.635969 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e505e446-b823-40ac-b126-8f800eafb710 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:49.650822 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 788/1596] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:54:49 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:54:49.652604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505e446-b823-40ac-b126-8f800eafb710 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:49.654394 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-50a1d9ee-ed3b-4c54-915a-603277d76694 None None] MFA Rules not processed for user `93ddf7b48ffe441eb5379ef455c3dd9e`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:49.655640 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-50a1d9ee-ed3b-4c54-915a-603277d76694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:49.664334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505e446-b823-40ac-b126-8f800eafb710 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:49.669902 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a12cc2-d716-4935-82e2-3fb1f56f0dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:49.670513 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a12cc2-d716-4935-82e2-3fb1f56f0dac admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:49.670912 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a12cc2-d716-4935-82e2-3fb1f56f0dac admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:49.671226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a12cc2-d716-4935-82e2-3fb1f56f0dac admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:49.672572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a12cc2-d716-4935-82e2-3fb1f56f0dac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:49.674244 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a12cc2-d716-4935-82e2-3fb1f56f0dac admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:49.685982 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 789/1597] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:54:49 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:54:49.690755 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a12cc2-d716-4935-82e2-3fb1f56f0dac admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:49.700695 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a12cc2-d716-4935-82e2-3fb1f56f0dac admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:49.706326 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3746e7a-6fcf-4bba-93e9-8155e56d5fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:49.710531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3746e7a-6fcf-4bba-93e9-8155e56d5fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:49.710531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3746e7a-6fcf-4bba-93e9-8155e56d5fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:49.710531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3746e7a-6fcf-4bba-93e9-8155e56d5fb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:49.710531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3746e7a-6fcf-4bba-93e9-8155e56d5fb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:49.710531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3746e7a-6fcf-4bba-93e9-8155e56d5fb1 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:49.724135 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 790/1598] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:54:49 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:54:49.724135 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3746e7a-6fcf-4bba-93e9-8155e56d5fb1 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:49.739979 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3746e7a-6fcf-4bba-93e9-8155e56d5fb1 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:49.740194 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 809/1599] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:49 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:49.740194 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1d9ee-ed3b-4c54-915a-603277d76694 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:49.743558 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147bd00-c459-495b-bd29-3f7d2685c654 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:49.744403 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147bd00-c459-495b-bd29-3f7d2685c654 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:49.744763 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147bd00-c459-495b-bd29-3f7d2685c654 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:49.745118 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e147bd00-c459-495b-bd29-3f7d2685c654 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:49.749411 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a1d9ee-ed3b-4c54-915a-603277d76694 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:49.754955 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b3d94-f13f-4851-b1a1-89f258a4c404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:49.755569 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147bd00-c459-495b-bd29-3f7d2685c654 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:49.755740 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b3d94-f13f-4851-b1a1-89f258a4c404 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:49.756003 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b3d94-f13f-4851-b1a1-89f258a4c404 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:49.756283 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b3d94-f13f-4851-b1a1-89f258a4c404 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:49.756982 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b3d94-f13f-4851-b1a1-89f258a4c404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:49.757086 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e147bd00-c459-495b-bd29-3f7d2685c654 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:49.758574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b3d94-f13f-4851-b1a1-89f258a4c404 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:49.767408 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 791/1600] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:54:49 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:54:49.768123 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147bd00-c459-495b-bd29-3f7d2685c654 service neutron] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:49.773227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 810/1601] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:54:49 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:54:49.774101 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b3d94-f13f-4851-b1a1-89f258a4c404 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:49.779040 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e147bd00-c459-495b-bd29-3f7d2685c654 service neutron] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:49.780411 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:54:49.784668 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b3d94-f13f-4851-b1a1-89f258a4c404 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:49.786312 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e97f8-c40f-4ac8-be89-5b908070b8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:49.787023 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e97f8-c40f-4ac8-be89-5b908070b8cc admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:49.787418 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e97f8-c40f-4ac8-be89-5b908070b8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:49.787748 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e97f8-c40f-4ac8-be89-5b908070b8cc admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:49.788542 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e97f8-c40f-4ac8-be89-5b908070b8cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:49.790240 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680ad59-980d-4ba7-9e04-1b168b14ba7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:49.790456 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e97f8-c40f-4ac8-be89-5b908070b8cc admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:49.791006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680ad59-980d-4ba7-9e04-1b168b14ba7c admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:49.794609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680ad59-980d-4ba7-9e04-1b168b14ba7c admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:49.794609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680ad59-980d-4ba7-9e04-1b168b14ba7c admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:49.794609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8680ad59-980d-4ba7-9e04-1b168b14ba7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:49.794609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8680ad59-980d-4ba7-9e04-1b168b14ba7c admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:49.810417 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 792/1602] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:54:49 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:54:49.810417 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e97f8-c40f-4ac8-be89-5b908070b8cc admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:49.810718 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 811/1603] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:54:49 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:54:49.810718 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680ad59-980d-4ba7-9e04-1b168b14ba7c admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:49.821469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680ad59-980d-4ba7-9e04-1b168b14ba7c admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:49.822269 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e97f8-c40f-4ac8-be89-5b908070b8cc admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:49.826755 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145db367-f852-43ed-96ec-6e658fd0395a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:49.828242 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145db367-f852-43ed-96ec-6e658fd0395a admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:49.828242 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145db367-f852-43ed-96ec-6e658fd0395a admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:49.828715 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145db367-f852-43ed-96ec-6e658fd0395a admin admin] PATH_INFO: `/v3/projects/89a4a0cf1852449e8bfc59ddb5a902c1/users/53bc14ada1564911b2f0a698aef3f333/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:49.829365 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c01fd8-d13e-4e5f-8ca6-b9e850325e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:49.830126 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c01fd8-d13e-4e5f-8ca6-b9e850325e85 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:49.830472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c01fd8-d13e-4e5f-8ca6-b9e850325e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:49.830818 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c01fd8-d13e-4e5f-8ca6-b9e850325e85 admin admin] PATH_INFO: `/v3/projects/9d121c6b6fc44c4ca99caf69e4f57436/users/1dddc20f605b461a897a661942f381d7/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:49.846293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145db367-f852-43ed-96ec-6e658fd0395a admin admin] RBAC: Authorizing `identity:create_grant(project_id=89a4a0cf1852449e8bfc59ddb5a902c1, user_id=53bc14ada1564911b2f0a698aef3f333, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:49.848813 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c01fd8-d13e-4e5f-8ca6-b9e850325e85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d121c6b6fc44c4ca99caf69e4f57436, user_id=1dddc20f605b461a897a661942f381d7, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:49.849114 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145db367-f852-43ed-96ec-6e658fd0395a admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:49.851705 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c01fd8-d13e-4e5f-8ca6-b9e850325e85 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:49.934776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 812/1605] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:54:49 2022] PUT /identity/v3/projects/89a4a0cf1852449e8bfc59ddb5a902c1/users/53bc14ada1564911b2f0a698aef3f333/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:54:49.934776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145db367-f852-43ed-96ec-6e658fd0395a admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:49.944436 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 793/1605] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:54:49 2022] PUT /identity/v3/projects/9d121c6b6fc44c4ca99caf69e4f57436/users/1dddc20f605b461a897a661942f381d7/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:54:49.945489 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c01fd8-d13e-4e5f-8ca6-b9e850325e85 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:50.012114 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145db367-f852-43ed-96ec-6e658fd0395a admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:50.017767 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388229c-49ee-411c-8473-50b92dda6d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:50.018403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388229c-49ee-411c-8473-50b92dda6d5a admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:50.018687 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388229c-49ee-411c-8473-50b92dda6d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:50.018962 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388229c-49ee-411c-8473-50b92dda6d5a admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:50.019774 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388229c-49ee-411c-8473-50b92dda6d5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:50.021719 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388229c-49ee-411c-8473-50b92dda6d5a admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:50.025089 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c01fd8-d13e-4e5f-8ca6-b9e850325e85 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:50.031780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac7e18-8b1d-4757-acc6-6fda8dfbd5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:50.032700 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac7e18-8b1d-4757-acc6-6fda8dfbd5dc admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:50.033105 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac7e18-8b1d-4757-acc6-6fda8dfbd5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:50.033445 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac7e18-8b1d-4757-acc6-6fda8dfbd5dc admin admin] PATH_INFO: `/v3/projects/98a5ee46ada143be8b05a26501f7a9d1/users/f92f95f0b4e947518d84dbc161938110/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:50.051846 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac7e18-8b1d-4757-acc6-6fda8dfbd5dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=98a5ee46ada143be8b05a26501f7a9d1, user_id=f92f95f0b4e947518d84dbc161938110, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:50.054585 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac7e18-8b1d-4757-acc6-6fda8dfbd5dc admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:50.099832 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 813/1606] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:54:49 2022] POST /identity/v3/projects => generated 388 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:54:50.099832 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388229c-49ee-411c-8473-50b92dda6d5a admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:50.130478 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 794/1607] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:54:49 2022] PUT /identity/v3/projects/98a5ee46ada143be8b05a26501f7a9d1/users/f92f95f0b4e947518d84dbc161938110/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:54:50.131521 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac7e18-8b1d-4757-acc6-6fda8dfbd5dc admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:50.175610 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388229c-49ee-411c-8473-50b92dda6d5a admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:50.205258 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:54:50.208912 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac7e18-8b1d-4757-acc6-6fda8dfbd5dc admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:50.214147 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4be10-7cdb-481e-8f2a-4ae3dfb41228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:50.214970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4be10-7cdb-481e-8f2a-4ae3dfb41228 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:50.215326 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4be10-7cdb-481e-8f2a-4ae3dfb41228 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:50.215686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4be10-7cdb-481e-8f2a-4ae3dfb41228 admin admin] PATH_INFO: `/v3/projects/1b3a294d5d02485980c82f065ee76bd7/users/ab1211acdd9a4df48c46f8807ed00063/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:50.232744 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff4be10-7cdb-481e-8f2a-4ae3dfb41228 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b3a294d5d02485980c82f065ee76bd7, user_id=ab1211acdd9a4df48c46f8807ed00063, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:50.236376 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff4be10-7cdb-481e-8f2a-4ae3dfb41228 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:50.245981 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa689e6-e202-4a00-84cd-62c8000c9c48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:50.250818 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa689e6-e202-4a00-84cd-62c8000c9c48 admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:50.250818 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa689e6-e202-4a00-84cd-62c8000c9c48 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:50.250818 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa689e6-e202-4a00-84cd-62c8000c9c48 admin admin] PATH_INFO: `/v3/projects/302e92110a1545f1b84644b8971bb6eb/users/f3f41f17f5154e01a699043edc350cf8/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:50.251518 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:54:50.264024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa689e6-e202-4a00-84cd-62c8000c9c48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=302e92110a1545f1b84644b8971bb6eb, user_id=f3f41f17f5154e01a699043edc350cf8, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:50.273620 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa689e6-e202-4a00-84cd-62c8000c9c48 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:50.324153 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 795/1608] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:54:50 2022] PUT /identity/v3/projects/1b3a294d5d02485980c82f065ee76bd7/users/ab1211acdd9a4df48c46f8807ed00063/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:54:50.329997 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeddfaa-cd3a-4195-9a52-dc5b70ec37db None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:50.330701 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeddfaa-cd3a-4195-9a52-dc5b70ec37db None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:50.330701 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeddfaa-cd3a-4195-9a52-dc5b70ec37db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:50.354731 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-faeddfaa-cd3a-4195-9a52-dc5b70ec37db None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:50.354731 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-faeddfaa-cd3a-4195-9a52-dc5b70ec37db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:50.357980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 814/1609] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:54:50 2022] PUT /identity/v3/projects/302e92110a1545f1b84644b8971bb6eb/users/f3f41f17f5154e01a699043edc350cf8/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:54:50.359582 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a8c82-6a8c-4748-b642-a798b1d636b2 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:50.360240 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a8c82-6a8c-4748-b642-a798b1d636b2 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:50.360592 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810a8c82-6a8c-4748-b642-a798b1d636b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:50.400352 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-810a8c82-6a8c-4748-b642-a798b1d636b2 None None] MFA Rules not processed for user `1dddc20f605b461a897a661942f381d7`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:50.401942 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-810a8c82-6a8c-4748-b642-a798b1d636b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:50.427595 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 796/1610] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:50.428359 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeddfaa-cd3a-4195-9a52-dc5b70ec37db None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:50.441909 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeddfaa-cd3a-4195-9a52-dc5b70ec37db None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:50.446872 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a24e1f2-692a-45a5-a26e-6a13f8a382a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:50.447523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24e1f2-692a-45a5-a26e-6a13f8a382a7 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:50.447801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24e1f2-692a-45a5-a26e-6a13f8a382a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:50.448066 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a24e1f2-692a-45a5-a26e-6a13f8a382a7 admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:50.448734 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a24e1f2-692a-45a5-a26e-6a13f8a382a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:50.450672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a24e1f2-692a-45a5-a26e-6a13f8a382a7 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:50.493382 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 815/1611] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:50.495730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869a0b2-c761-4abc-9c8f-91f10b9ea70d None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:50.496597 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869a0b2-c761-4abc-9c8f-91f10b9ea70d None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:50.497007 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869a0b2-c761-4abc-9c8f-91f10b9ea70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:50.531257 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 797/1612] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:54:50 2022] POST /identity/v3/users => generated 301 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:54:50.533007 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe93c38-5e7f-4000-959c-18abcd773847 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:50.533300 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe93c38-5e7f-4000-959c-18abcd773847 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:50.534717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe93c38-5e7f-4000-959c-18abcd773847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:50.555189 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-5869a0b2-c761-4abc-9c8f-91f10b9ea70d None None] MFA Rules not processed for user `f92f95f0b4e947518d84dbc161938110`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:50.556423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-5869a0b2-c761-4abc-9c8f-91f10b9ea70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:50.571990 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-2fe93c38-5e7f-4000-959c-18abcd773847 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:50.573542 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-2fe93c38-5e7f-4000-959c-18abcd773847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:50.583470 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 798/1613] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:50.586653 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0daed-c5ec-4717-9816-ab3d4ebb4daf None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:50.586653 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0daed-c5ec-4717-9816-ab3d4ebb4daf None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:50.586653 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf0daed-c5ec-4717-9816-ab3d4ebb4daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:50.615766 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-abf0daed-c5ec-4717-9816-ab3d4ebb4daf None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:50.615766 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-abf0daed-c5ec-4717-9816-ab3d4ebb4daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:50.621366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 799/1614] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:50.625204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0daed-c5ec-4717-9816-ab3d4ebb4daf None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:50.642476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 816/1615] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:50.644649 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bab03f-2c7b-410b-9e61-e972a7d49a4c None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:50.644649 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bab03f-2c7b-410b-9e61-e972a7d49a4c None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:50.645033 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bab03f-2c7b-410b-9e61-e972a7d49a4c None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:50.646673 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 817/1616] 10.222.0.115 () {54 vars in 967 bytes} [Tue Aug 2 10:54:50 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:54:50.658572 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bab03f-2c7b-410b-9e61-e972a7d49a4c None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:50.684190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bab03f-2c7b-410b-9e61-e972a7d49a4c None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:50.691791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc1f539-0647-40e0-9083-eda8273f27ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:50.693355 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1f539-0647-40e0-9083-eda8273f27ce admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:50.694129 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1f539-0647-40e0-9083-eda8273f27ce admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:50.694650 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc1f539-0647-40e0-9083-eda8273f27ce admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:50.696109 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc1f539-0647-40e0-9083-eda8273f27ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:50.698846 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc1f539-0647-40e0-9083-eda8273f27ce admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:50.702362 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf0daed-c5ec-4717-9816-ab3d4ebb4daf None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:50.707121 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b69484-c288-43cd-b3ff-13a6396257d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:50.709072 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b69484-c288-43cd-b3ff-13a6396257d7 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:50.709072 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b69484-c288-43cd-b3ff-13a6396257d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:50.709072 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b69484-c288-43cd-b3ff-13a6396257d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:50.714070 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 818/1617] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:54:50 2022] GET /identity/v3/roles => generated 1427 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:54:50.715279 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc1f539-0647-40e0-9083-eda8273f27ce admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:50.718572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b69484-c288-43cd-b3ff-13a6396257d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:50.720104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b69484-c288-43cd-b3ff-13a6396257d7 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:50.727578 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc1f539-0647-40e0-9083-eda8273f27ce admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:50.731481 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dce225a-5dc6-439f-be3b-58bd10c847e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:50.736157 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 800/1618] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:54:50 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:54:50.736407 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce225a-5dc6-439f-be3b-58bd10c847e6 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:50.736407 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce225a-5dc6-439f-be3b-58bd10c847e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:50.736407 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dce225a-5dc6-439f-be3b-58bd10c847e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:50.736696 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b69484-c288-43cd-b3ff-13a6396257d7 service neutron] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:50.743006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b69484-c288-43cd-b3ff-13a6396257d7 service neutron] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:50.744881 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dce225a-5dc6-439f-be3b-58bd10c847e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:50.746646 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dce225a-5dc6-439f-be3b-58bd10c847e6 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:50.748081 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b58e6a-87e9-43d8-b457-0ef5e9a62a0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:50.748868 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b58e6a-87e9-43d8-b457-0ef5e9a62a0e service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:50.749409 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b58e6a-87e9-43d8-b457-0ef5e9a62a0e service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:50.750436 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b58e6a-87e9-43d8-b457-0ef5e9a62a0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:50.760202 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 819/1619] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:54:50 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:54:50.761135 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b58e6a-87e9-43d8-b457-0ef5e9a62a0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:50.762258 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd0182-ad15-4e13-bcfa-c3d0a6fc69dc None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:50.762605 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd0182-ad15-4e13-bcfa-c3d0a6fc69dc None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:50.766606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd0182-ad15-4e13-bcfa-c3d0a6fc69dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:50.766752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b58e6a-87e9-43d8-b457-0ef5e9a62a0e service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:50.775651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 801/1620] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:54:50 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:54:50.776952 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da94ab-baef-4d3e-b16f-ac77c92d854e None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:50.777244 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da94ab-baef-4d3e-b16f-ac77c92d854e None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:50.777520 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4da94ab-baef-4d3e-b16f-ac77c92d854e None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:50.780269 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 802/1621] 10.222.0.115 () {54 vars in 967 bytes} [Tue Aug 2 10:54:50 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:54:50.781631 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4da94ab-baef-4d3e-b16f-ac77c92d854e None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:50.797363 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4da94ab-baef-4d3e-b16f-ac77c92d854e None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:50.805727 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c282d7-910d-4808-bccc-fa0c313123b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:50.806916 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c282d7-910d-4808-bccc-fa0c313123b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:50.807650 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c282d7-910d-4808-bccc-fa0c313123b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:50.808141 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c282d7-910d-4808-bccc-fa0c313123b7 admin admin] PATH_INFO: `/v3/projects/9ac21ce7806d435991ede2da815318ee/users/180dc234f4a14a6db15ec765ba5f4084/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:50.822727 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-05bd0182-ad15-4e13-bcfa-c3d0a6fc69dc None None] MFA Rules not processed for user `d0478bd540f446ec810e2091c46ac49c`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:50.824270 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-05bd0182-ad15-4e13-bcfa-c3d0a6fc69dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:50.830878 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c282d7-910d-4808-bccc-fa0c313123b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ac21ce7806d435991ede2da815318ee, user_id=180dc234f4a14a6db15ec765ba5f4084, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:50.834862 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c282d7-910d-4808-bccc-fa0c313123b7 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:50.856232 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:54:50.924090 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 820/1622] 10.222.0.115 () {58 vars in 1064 bytes} [Tue Aug 2 10:54:50 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:50.925527 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e8431-1b4d-4c80-906f-ab589f78d13a None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:50.925826 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e8431-1b4d-4c80-906f-ab589f78d13a None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:50.926106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e8431-1b4d-4c80-906f-ab589f78d13a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:50.932972 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 803/1623] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:54:50 2022] PUT /identity/v3/projects/9ac21ce7806d435991ede2da815318ee/users/180dc234f4a14a6db15ec765ba5f4084/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:54:50.934053 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c282d7-910d-4808-bccc-fa0c313123b7 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:50.951025 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-ca2e8431-1b4d-4c80-906f-ab589f78d13a None None] MFA Rules not processed for user `d0478bd540f446ec810e2091c46ac49c`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:50.952740 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-ca2e8431-1b4d-4c80-906f-ab589f78d13a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:50.961293 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:54:51.025450 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c282d7-910d-4808-bccc-fa0c313123b7 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:51.032701 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c959a2c-b48f-499c-96a1-8cb15a777704 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:51.032906 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c959a2c-b48f-499c-96a1-8cb15a777704 service nova] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:51.033106 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c959a2c-b48f-499c-96a1-8cb15a777704 service nova] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:51.033378 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c959a2c-b48f-499c-96a1-8cb15a777704 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:51.055252 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 821/1624] 10.222.0.115 () {58 vars in 1064 bytes} [Tue Aug 2 10:54:50 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:51.057071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e8431-1b4d-4c80-906f-ab589f78d13a None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:51.105825 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c959a2c-b48f-499c-96a1-8cb15a777704 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:51.108676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c959a2c-b48f-499c-96a1-8cb15a777704 service nova] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:51.122561 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 804/1625] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:54:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:54:51.125198 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c959a2c-b48f-499c-96a1-8cb15a777704 service nova] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:51.140433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c959a2c-b48f-499c-96a1-8cb15a777704 service nova] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:51.147881 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f2702-8c08-409c-bedc-c349d8dc0217 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:51.148584 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f2702-8c08-409c-bedc-c349d8dc0217 service nova] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:51.148929 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f2702-8c08-409c-bedc-c349d8dc0217 service nova] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:51.149524 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f2702-8c08-409c-bedc-c349d8dc0217 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:51.220565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e8431-1b4d-4c80-906f-ab589f78d13a None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:51.233869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de2cc4-ae7b-4179-90ab-b23126196702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:51.238588 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de2cc4-ae7b-4179-90ab-b23126196702 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:51.239534 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de2cc4-ae7b-4179-90ab-b23126196702 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:51.240300 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49de2cc4-ae7b-4179-90ab-b23126196702 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:51.241384 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de2cc4-ae7b-4179-90ab-b23126196702 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:51.243582 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49de2cc4-ae7b-4179-90ab-b23126196702 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:51.265384 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f2702-8c08-409c-bedc-c349d8dc0217 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:51.279974 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f2702-8c08-409c-bedc-c349d8dc0217 service nova] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:51.296069 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 822/1626] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:54:51 2022] GET /identity/v3/roles => generated 1427 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:54:51.301331 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de2cc4-ae7b-4179-90ab-b23126196702 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:51.311883 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 805/1627] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:54:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:54:51.337053 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49de2cc4-ae7b-4179-90ab-b23126196702 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:51.354945 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5e0328-4b15-4698-bf33-d6c9d89ebf8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:51.356148 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e0328-4b15-4698-bf33-d6c9d89ebf8b admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:51.357094 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e0328-4b15-4698-bf33-d6c9d89ebf8b admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:51.357623 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5e0328-4b15-4698-bf33-d6c9d89ebf8b admin admin] PATH_INFO: `/v3/projects/9ac21ce7806d435991ede2da815318ee/users/180dc234f4a14a6db15ec765ba5f4084/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:51.360313 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e0328-4b15-4698-bf33-d6c9d89ebf8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ac21ce7806d435991ede2da815318ee, user_id=180dc234f4a14a6db15ec765ba5f4084, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:51.365274 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5e0328-4b15-4698-bf33-d6c9d89ebf8b admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:51.381396 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 823/1628] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:54:51 2022] PUT /identity/v3/projects/9ac21ce7806d435991ede2da815318ee/users/180dc234f4a14a6db15ec765ba5f4084/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:54:51.392305 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad0472-8c7c-479b-9453-62f5398314f7 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:51.392833 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad0472-8c7c-479b-9453-62f5398314f7 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:51.392833 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ad0472-8c7c-479b-9453-62f5398314f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:51.469433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-c4ad0472-8c7c-479b-9453-62f5398314f7 None None] MFA Rules not processed for user `180dc234f4a14a6db15ec765ba5f4084`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:51.471146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-c4ad0472-8c7c-479b-9453-62f5398314f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:51.565968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:54:51.637992 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 806/1629] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:51 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:54:51.643385 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1f8db-3f24-4286-a0a2-4c93f8f0771f None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:51.643385 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1f8db-3f24-4286-a0a2-4c93f8f0771f None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:51.643385 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1f8db-3f24-4286-a0a2-4c93f8f0771f None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:51.643385 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 824/1630] 10.222.0.115 () {54 vars in 917 bytes} [Tue Aug 2 10:54:51 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:54:51.651433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad0472-8c7c-479b-9453-62f5398314f7 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:51.654176 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8878706-fd95-4229-b5b9-710dd7357765 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:51.654468 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8878706-fd95-4229-b5b9-710dd7357765 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:51.654746 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8878706-fd95-4229-b5b9-710dd7357765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:51.728696 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-b8878706-fd95-4229-b5b9-710dd7357765 None None] MFA Rules not processed for user `d0478bd540f446ec810e2091c46ac49c`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:51.729307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-b8878706-fd95-4229-b5b9-710dd7357765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:51.817288 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ad0472-8c7c-479b-9453-62f5398314f7 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:51.817288 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc620a-135b-400a-98fb-3437fc53bb32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:51.817288 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc620a-135b-400a-98fb-3437fc53bb32 service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:51.817288 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc620a-135b-400a-98fb-3437fc53bb32 service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:51.817710 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc620a-135b-400a-98fb-3437fc53bb32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:51.822598 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:54:51.826637 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc620a-135b-400a-98fb-3437fc53bb32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:51.829532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc620a-135b-400a-98fb-3437fc53bb32 service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:51.850526 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 807/1631] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:54:51 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:54:51.852987 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a9da7-acec-4f1e-a650-c4dd23ab8ece None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:51.866544 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a9da7-acec-4f1e-a650-c4dd23ab8ece None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:51.866902 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a9da7-acec-4f1e-a650-c4dd23ab8ece None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:51.869426 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 808/1632] 10.222.0.115 () {54 vars in 917 bytes} [Tue Aug 2 10:54:51 2022] GET /identity => generated 271 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:54:51.875275 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403be22f-c25e-4c78-b0bd-622df7f68c76 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:51.875275 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403be22f-c25e-4c78-b0bd-622df7f68c76 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:51.875275 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403be22f-c25e-4c78-b0bd-622df7f68c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:51.935970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-403be22f-c25e-4c78-b0bd-622df7f68c76 None None] MFA Rules not processed for user `b7fc4b12f4f541609ddf148c8b51d8f1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:51.937754 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-403be22f-c25e-4c78-b0bd-622df7f68c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:51.943168 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 825/1633] 10.222.0.115 () {58 vars in 1014 bytes} [Tue Aug 2 10:54:51 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:51.954644 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7ba03-ce96-4e2f-89fe-81088aac0056 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:51.957514 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7ba03-ce96-4e2f-89fe-81088aac0056 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:51.960891 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b7ba03-ce96-4e2f-89fe-81088aac0056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:52.005247 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-10b7ba03-ce96-4e2f-89fe-81088aac0056 None None] MFA Rules not processed for user `d0478bd540f446ec810e2091c46ac49c`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:52.008882 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-10b7ba03-ce96-4e2f-89fe-81088aac0056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:52.022058 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 809/1634] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:51 2022] POST /identity/v3/auth/tokens => generated 3290 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:52.023285 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403be22f-c25e-4c78-b0bd-622df7f68c76 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:52.035306 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 826/1635] 10.222.0.115 () {58 vars in 1014 bytes} [Tue Aug 2 10:54:51 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:52.038054 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7ba03-ce96-4e2f-89fe-81088aac0056 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:52.058309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b7ba03-ce96-4e2f-89fe-81088aac0056 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:52.070338 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab27a1-4315-4818-b7a8-9f89da8070c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:52.070785 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab27a1-4315-4818-b7a8-9f89da8070c6 service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:52.071089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab27a1-4315-4818-b7a8-9f89da8070c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:52.071378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab27a1-4315-4818-b7a8-9f89da8070c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:52.096839 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ab27a1-4315-4818-b7a8-9f89da8070c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:52.099517 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ab27a1-4315-4818-b7a8-9f89da8070c6 service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:52.117081 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403be22f-c25e-4c78-b0bd-622df7f68c76 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:52.122927 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8debf9c0-b34a-4b96-a3cd-9086ea623de7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:52.124068 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8debf9c0-b34a-4b96-a3cd-9086ea623de7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:52.124511 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8debf9c0-b34a-4b96-a3cd-9086ea623de7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:52.124899 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8debf9c0-b34a-4b96-a3cd-9086ea623de7 admin admin] PATH_INFO: `/v3/users/93ddf7b48ffe441eb5379ef455c3dd9e` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:52.126780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8debf9c0-b34a-4b96-a3cd-9086ea623de7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=93ddf7b48ffe441eb5379ef455c3dd9e)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:52.128800 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 827/1636] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:54:52 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:54:52.129726 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8debf9c0-b34a-4b96-a3cd-9086ea623de7 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:52.140595 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab27a1-4315-4818-b7a8-9f89da8070c6 service cinder] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:52.145466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:54:52.176926 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab27a1-4315-4818-b7a8-9f89da8070c6 service cinder] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:52.189689 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076bb04b-f907-474a-9f18-f680e04f1f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:52.191420 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076bb04b-f907-474a-9f18-f680e04f1f0f admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:52.191710 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076bb04b-f907-474a-9f18-f680e04f1f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:52.191989 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076bb04b-f907-474a-9f18-f680e04f1f0f admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:52.195472 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076bb04b-f907-474a-9f18-f680e04f1f0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:52.198922 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076bb04b-f907-474a-9f18-f680e04f1f0f admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:52.359171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 828/1637] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:54:52 2022] POST /identity/v3/projects => generated 404 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:54:52.360571 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076bb04b-f907-474a-9f18-f680e04f1f0f admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:52.371728 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-8debf9c0-b34a-4b96-a3cd-9086ea623de7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93ddf7b48ffe441eb5379ef455c3dd9e'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:52.388140 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-8debf9c0-b34a-4b96-a3cd-9086ea623de7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93ddf7b48ffe441eb5379ef455c3dd9e'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:52.389538 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-8debf9c0-b34a-4b96-a3cd-9086ea623de7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93ddf7b48ffe441eb5379ef455c3dd9e'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:52.471858 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-8debf9c0-b34a-4b96-a3cd-9086ea623de7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93ddf7b48ffe441eb5379ef455c3dd9e'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:52.495998 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-8debf9c0-b34a-4b96-a3cd-9086ea623de7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93ddf7b48ffe441eb5379ef455c3dd9e'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:52.514677 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 810/1638] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:54:52 2022] DELETE /identity/v3/users/93ddf7b48ffe441eb5379ef455c3dd9e => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:54:52.514992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076bb04b-f907-474a-9f18-f680e04f1f0f admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:52.515277 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8debf9c0-b34a-4b96-a3cd-9086ea623de7 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:52.518597 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-8debf9c0-b34a-4b96-a3cd-9086ea623de7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:52.630126 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c544360-9562-48d1-aa36-a1e6e05245ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:52.631734 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c544360-9562-48d1-aa36-a1e6e05245ab service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:52.632023 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c544360-9562-48d1-aa36-a1e6e05245ab service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:52.632296 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c544360-9562-48d1-aa36-a1e6e05245ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:52.641581 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-8c544360-9562-48d1-aa36-a1e6e05245ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:52.664283 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8debf9c0-b34a-4b96-a3cd-9086ea623de7 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:52.670464 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feac226d-b4cd-4ae9-9323-0c312ed291d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:52.672688 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac226d-b4cd-4ae9-9323-0c312ed291d9 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:52.673186 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac226d-b4cd-4ae9-9323-0c312ed291d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:52.679841 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac226d-b4cd-4ae9-9323-0c312ed291d9 admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:52.679841 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac226d-b4cd-4ae9-9323-0c312ed291d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:52.679841 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feac226d-b4cd-4ae9-9323-0c312ed291d9 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:52.777502 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:54:52.792274 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 811/1639] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:54:52 2022] POST /identity/v3/users => generated 309 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:54:52.793859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aed08e-1aa3-4c91-8d41-7744a2b07a6b None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:52.795355 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aed08e-1aa3-4c91-8d41-7744a2b07a6b None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:52.795702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68aed08e-1aa3-4c91-8d41-7744a2b07a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:52.840088 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-68aed08e-1aa3-4c91-8d41-7744a2b07a6b None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:52.841214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c544360-9562-48d1-aa36-a1e6e05245ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:52.841466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-68aed08e-1aa3-4c91-8d41-7744a2b07a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:52.843572 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c544360-9562-48d1-aa36-a1e6e05245ab service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:52.855611 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 812/1640] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:52.855726 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aed08e-1aa3-4c91-8d41-7744a2b07a6b None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:52.867284 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 829/1641] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:54:52 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:54:52.868216 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c544360-9562-48d1-aa36-a1e6e05245ab service cinder] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:52.869832 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68aed08e-1aa3-4c91-8d41-7744a2b07a6b None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:52.870177 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-8c544360-9562-48d1-aa36-a1e6e05245ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:53.060501 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3caef1d-9bc4-421e-80b0-f34bbc0ebe78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:53.060501 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3caef1d-9bc4-421e-80b0-f34bbc0ebe78 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:53.060501 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3caef1d-9bc4-421e-80b0-f34bbc0ebe78 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:53.060501 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3caef1d-9bc4-421e-80b0-f34bbc0ebe78 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:53.069051 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3caef1d-9bc4-421e-80b0-f34bbc0ebe78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:53.069051 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3caef1d-9bc4-421e-80b0-f34bbc0ebe78 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:53.069051 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 813/1642] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:54:52 2022] GET /identity/v3/roles => generated 1427 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:54:53.069051 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3caef1d-9bc4-421e-80b0-f34bbc0ebe78 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:53.069051 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3caef1d-9bc4-421e-80b0-f34bbc0ebe78 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:53.075417 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e119a-2080-4965-b830-2b7acb4bd385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:53.075417 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e119a-2080-4965-b830-2b7acb4bd385 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:53.075417 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e119a-2080-4965-b830-2b7acb4bd385 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:53.075417 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e119a-2080-4965-b830-2b7acb4bd385 admin admin] PATH_INFO: `/v3/projects/93d71cb6f13b470783204fc17f7a03c4/users/fbf5248a340944bb831f446f97426409/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:53.082181 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e119a-2080-4965-b830-2b7acb4bd385 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93d71cb6f13b470783204fc17f7a03c4, user_id=fbf5248a340944bb831f446f97426409, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:53.082181 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e119a-2080-4965-b830-2b7acb4bd385 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:53.087246 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c544360-9562-48d1-aa36-a1e6e05245ab service cinder] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:53.087246 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b6c421-9ba0-4f0f-88a9-458954efbb3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:53.087246 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b6c421-9ba0-4f0f-88a9-458954efbb3f service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:53.087246 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b6c421-9ba0-4f0f-88a9-458954efbb3f service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:53.090864 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b6c421-9ba0-4f0f-88a9-458954efbb3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:53.094927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b6c421-9ba0-4f0f-88a9-458954efbb3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:53.105308 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b6c421-9ba0-4f0f-88a9-458954efbb3f service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:53.141711 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 830/1643] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:54:52 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:54:53.191943 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 814/1644] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:54:52 2022] PUT /identity/v3/projects/93d71cb6f13b470783204fc17f7a03c4/users/fbf5248a340944bb831f446f97426409/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:54:53.197369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b6c421-9ba0-4f0f-88a9-458954efbb3f service neutron] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:53.201809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e119a-2080-4965-b830-2b7acb4bd385 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:53.207184 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b6c421-9ba0-4f0f-88a9-458954efbb3f service neutron] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:53.208786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-fb4e119a-2080-4965-b830-2b7acb4bd385 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:53.296430 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:DELETE=10,keystone:SELECT=85,keystone:INSERT=5 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:54:53.310281 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072de56-5876-4ef9-9bef-ff5db1368053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:53.310941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072de56-5876-4ef9-9bef-ff5db1368053 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:53.311248 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072de56-5876-4ef9-9bef-ff5db1368053 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:53.311526 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072de56-5876-4ef9-9bef-ff5db1368053 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:53.314466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6072de56-5876-4ef9-9bef-ff5db1368053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:53.322729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6072de56-5876-4ef9-9bef-ff5db1368053 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:53.338311 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 831/1645] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:54:53 2022] GET /identity/v3/roles => generated 1427 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:54:53.339502 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072de56-5876-4ef9-9bef-ff5db1368053 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:53.342594 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e119a-2080-4965-b830-2b7acb4bd385 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:53.352990 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300d108-3f1f-4310-867e-f5d3992fbf85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:53.354949 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300d108-3f1f-4310-867e-f5d3992fbf85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:53.355588 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300d108-3f1f-4310-867e-f5d3992fbf85 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:53.355995 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6300d108-3f1f-4310-867e-f5d3992fbf85 admin admin] PATH_INFO: `/v3/users/f92f95f0b4e947518d84dbc161938110` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:53.357455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300d108-3f1f-4310-867e-f5d3992fbf85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f92f95f0b4e947518d84dbc161938110)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:53.358358 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072de56-5876-4ef9-9bef-ff5db1368053 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:53.361145 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6300d108-3f1f-4310-867e-f5d3992fbf85 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:53.380332 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e54be-485f-49d7-81e8-39bc06d4f72e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:53.381630 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e54be-485f-49d7-81e8-39bc06d4f72e admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:53.382054 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e54be-485f-49d7-81e8-39bc06d4f72e admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:53.383546 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e54be-485f-49d7-81e8-39bc06d4f72e admin admin] PATH_INFO: `/v3/projects/93d71cb6f13b470783204fc17f7a03c4/users/fbf5248a340944bb831f446f97426409/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:53.389032 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929e54be-485f-49d7-81e8-39bc06d4f72e admin admin] RBAC: Authorizing `identity:create_grant(project_id=93d71cb6f13b470783204fc17f7a03c4, user_id=fbf5248a340944bb831f446f97426409, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:53.393625 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929e54be-485f-49d7-81e8-39bc06d4f72e admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:53.439391 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 832/1646] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:54:53 2022] PUT /identity/v3/projects/93d71cb6f13b470783204fc17f7a03c4/users/fbf5248a340944bb831f446f97426409/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:54:53.583673 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-6300d108-3f1f-4310-867e-f5d3992fbf85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f92f95f0b4e947518d84dbc161938110'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:53.614662 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-6300d108-3f1f-4310-867e-f5d3992fbf85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f92f95f0b4e947518d84dbc161938110'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:53.615806 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-6300d108-3f1f-4310-867e-f5d3992fbf85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f92f95f0b4e947518d84dbc161938110'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:53.706617 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-6300d108-3f1f-4310-867e-f5d3992fbf85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f92f95f0b4e947518d84dbc161938110'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:53.740543 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-6300d108-3f1f-4310-867e-f5d3992fbf85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f92f95f0b4e947518d84dbc161938110'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:53.767068 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 815/1647] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:54:53 2022] DELETE /identity/v3/users/f92f95f0b4e947518d84dbc161938110 => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:54:53.771737 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e66e6-ddcd-4859-96c8-da50ce69a773 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:53.772154 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e66e6-ddcd-4859-96c8-da50ce69a773 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:53.772466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1e66e6-ddcd-4859-96c8-da50ce69a773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:53.807497 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-0f1e66e6-ddcd-4859-96c8-da50ce69a773 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:53.809399 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-0f1e66e6-ddcd-4859-96c8-da50ce69a773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:53.900861 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 833/1648] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:53 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:53.918569 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300d108-3f1f-4310-867e-f5d3992fbf85 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:53.920073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-6300d108-3f1f-4310-867e-f5d3992fbf85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:54.034977 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6300d108-3f1f-4310-867e-f5d3992fbf85 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:54.038718 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fc45f-26ca-46e2-8017-afadc806ddeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:54.040620 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fc45f-26ca-46e2-8017-afadc806ddeb service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:54.040903 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fc45f-26ca-46e2-8017-afadc806ddeb service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:54.041178 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3fc45f-26ca-46e2-8017-afadc806ddeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:54.073059 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3fc45f-26ca-46e2-8017-afadc806ddeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:54.079140 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3fc45f-26ca-46e2-8017-afadc806ddeb service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:54.111448 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 816/1649] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:54:53 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:54:54.429990 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e66e6-ddcd-4859-96c8-da50ce69a773 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:54.436294 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-0f1e66e6-ddcd-4859-96c8-da50ce69a773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:54.473288 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1e66e6-ddcd-4859-96c8-da50ce69a773 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:54.499384 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f44f1-b527-4eff-abac-fc50655963f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:54.500654 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f44f1-b527-4eff-abac-fc50655963f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:54.501415 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f44f1-b527-4eff-abac-fc50655963f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:54.502017 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253f44f1-b527-4eff-abac-fc50655963f4 admin admin] PATH_INFO: `/v3/projects/6d0f5621fd55478ca86c8fc95ddb48bf` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:54.505379 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f44f1-b527-4eff-abac-fc50655963f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d0f5621fd55478ca86c8fc95ddb48bf)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:54.509451 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253f44f1-b527-4eff-abac-fc50655963f4 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:54.511983 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:54:54.685442 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-253f44f1-b527-4eff-abac-fc50655963f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d0f5621fd55478ca86c8fc95ddb48bf', 'request_id': 'req-253f44f1-b527-4eff-abac-fc50655963f4'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:54.700819 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-253f44f1-b527-4eff-abac-fc50655963f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d0f5621fd55478ca86c8fc95ddb48bf', 'request_id': 'req-253f44f1-b527-4eff-abac-fc50655963f4'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:54.758335 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-253f44f1-b527-4eff-abac-fc50655963f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d0f5621fd55478ca86c8fc95ddb48bf', 'request_id': 'req-253f44f1-b527-4eff-abac-fc50655963f4'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:54.782271 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-253f44f1-b527-4eff-abac-fc50655963f4 admin admin] The token cache is being invalidate because project 6d0f5621fd55478ca86c8fc95ddb48bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:54:54.782850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-253f44f1-b527-4eff-abac-fc50655963f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:54.784145 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-253f44f1-b527-4eff-abac-fc50655963f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:54.790852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 834/1650] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:54:54 2022] DELETE /identity/v3/projects/6d0f5621fd55478ca86c8fc95ddb48bf => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:54:54.791774 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fc45f-26ca-46e2-8017-afadc806ddeb service neutron] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:54.816650 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3fc45f-26ca-46e2-8017-afadc806ddeb service neutron] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:54.943797 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed1eff-40ac-41d1-9c2c-ea7557eb8ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:54.944466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed1eff-40ac-41d1-9c2c-ea7557eb8ac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:54.944795 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed1eff-40ac-41d1-9c2c-ea7557eb8ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:54.945080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffed1eff-40ac-41d1-9c2c-ea7557eb8ac6 admin admin] PATH_INFO: `/v3/users/180dc234f4a14a6db15ec765ba5f4084` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:54.947523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed1eff-40ac-41d1-9c2c-ea7557eb8ac6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=180dc234f4a14a6db15ec765ba5f4084)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:54.951654 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffed1eff-40ac-41d1-9c2c-ea7557eb8ac6 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:55.217437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ffed1eff-40ac-41d1-9c2c-ea7557eb8ac6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '180dc234f4a14a6db15ec765ba5f4084'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:55.237289 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ffed1eff-40ac-41d1-9c2c-ea7557eb8ac6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '180dc234f4a14a6db15ec765ba5f4084'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:55.238079 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ffed1eff-40ac-41d1-9c2c-ea7557eb8ac6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '180dc234f4a14a6db15ec765ba5f4084'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:55.255982 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:54:55.324388 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ffed1eff-40ac-41d1-9c2c-ea7557eb8ac6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '180dc234f4a14a6db15ec765ba5f4084'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:55.345028 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ffed1eff-40ac-41d1-9c2c-ea7557eb8ac6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '180dc234f4a14a6db15ec765ba5f4084'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:55.421430 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 817/1651] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:54:54 2022] DELETE /identity/v3/users/180dc234f4a14a6db15ec765ba5f4084 => generated 0 bytes in 630 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:54:56.088932 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f44f1-b527-4eff-abac-fc50655963f4 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:56.091252 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-253f44f1-b527-4eff-abac-fc50655963f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:56.127817 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253f44f1-b527-4eff-abac-fc50655963f4 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:56.325362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c8f7e6-3405-4312-8ef4-befee67f7bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:56.326576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8f7e6-3405-4312-8ef4-befee67f7bbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:56.327006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8f7e6-3405-4312-8ef4-befee67f7bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:56.328041 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c8f7e6-3405-4312-8ef4-befee67f7bbe admin admin] PATH_INFO: `/v3/projects/98a5ee46ada143be8b05a26501f7a9d1` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:56.332969 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c8f7e6-3405-4312-8ef4-befee67f7bbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=98a5ee46ada143be8b05a26501f7a9d1)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:56.335600 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c8f7e6-3405-4312-8ef4-befee67f7bbe admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:56.520825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-26c8f7e6-3405-4312-8ef4-befee67f7bbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98a5ee46ada143be8b05a26501f7a9d1', 'request_id': 'req-26c8f7e6-3405-4312-8ef4-befee67f7bbe'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:56.553011 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-26c8f7e6-3405-4312-8ef4-befee67f7bbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98a5ee46ada143be8b05a26501f7a9d1', 'request_id': 'req-26c8f7e6-3405-4312-8ef4-befee67f7bbe'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:56.582624 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed1eff-40ac-41d1-9c2c-ea7557eb8ac6 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:56.586158 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-ffed1eff-40ac-41d1-9c2c-ea7557eb8ac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:56.610482 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffed1eff-40ac-41d1-9c2c-ea7557eb8ac6 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:56.626155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-26c8f7e6-3405-4312-8ef4-befee67f7bbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98a5ee46ada143be8b05a26501f7a9d1', 'request_id': 'req-26c8f7e6-3405-4312-8ef4-befee67f7bbe'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:56.648071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-26c8f7e6-3405-4312-8ef4-befee67f7bbe admin admin] The token cache is being invalidate because project 98a5ee46ada143be8b05a26501f7a9d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:54:56.648364 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-26c8f7e6-3405-4312-8ef4-befee67f7bbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:56.650667 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-26c8f7e6-3405-4312-8ef4-befee67f7bbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:56.659919 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 835/1652] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:54:56 2022] DELETE /identity/v3/projects/98a5ee46ada143be8b05a26501f7a9d1 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:54:56.661605 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f52ee3-4684-46f8-8ded-d090d2a895a4 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:56.661895 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f52ee3-4684-46f8-8ded-d090d2a895a4 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:56.662179 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f52ee3-4684-46f8-8ded-d090d2a895a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:56.676664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5598aad-3fc4-4f07-be6f-aec74fa108ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:56.678917 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5598aad-3fc4-4f07-be6f-aec74fa108ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:56.678917 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5598aad-3fc4-4f07-be6f-aec74fa108ed admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:56.678917 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5598aad-3fc4-4f07-be6f-aec74fa108ed admin admin] PATH_INFO: `/v3/users/1dddc20f605b461a897a661942f381d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:56.679963 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5598aad-3fc4-4f07-be6f-aec74fa108ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=1dddc20f605b461a897a661942f381d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:56.682433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5598aad-3fc4-4f07-be6f-aec74fa108ed admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:56.724550 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-07f52ee3-4684-46f8-8ded-d090d2a895a4 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:56.725796 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-07f52ee3-4684-46f8-8ded-d090d2a895a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:56.755967 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 836/1653] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:56 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:54:56.757770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966fe53c-76f1-4612-85f1-c82e98528fb3 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:56.758173 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966fe53c-76f1-4612-85f1-c82e98528fb3 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:56.758603 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966fe53c-76f1-4612-85f1-c82e98528fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:56.792770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-966fe53c-76f1-4612-85f1-c82e98528fb3 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:56.796882 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-966fe53c-76f1-4612-85f1-c82e98528fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:56.816357 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 837/1654] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:56 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:54:56.817594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966fe53c-76f1-4612-85f1-c82e98528fb3 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:56.860679 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966fe53c-76f1-4612-85f1-c82e98528fb3 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:56.945935 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a2fd3-b06e-4251-9c54-5571436fcb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:56.945935 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a2fd3-b06e-4251-9c54-5571436fcb34 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:56.945935 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a2fd3-b06e-4251-9c54-5571436fcb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:56.945935 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889a2fd3-b06e-4251-9c54-5571436fcb34 admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:56.947415 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a2fd3-b06e-4251-9c54-5571436fcb34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:54:56.947415 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a2fd3-b06e-4251-9c54-5571436fcb34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:56.947415 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889a2fd3-b06e-4251-9c54-5571436fcb34 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:56.947415 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:54:56.947415 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 838/1655] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:54:56 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:54:56.947415 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a2fd3-b06e-4251-9c54-5571436fcb34 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:56.962492 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889a2fd3-b06e-4251-9c54-5571436fcb34 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:56.977560 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1145b1-bbed-47fd-9b5e-0a1f4c69da0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:56.978208 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1145b1-bbed-47fd-9b5e-0a1f4c69da0b admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:56.979077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1145b1-bbed-47fd-9b5e-0a1f4c69da0b admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:56.979077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1145b1-bbed-47fd-9b5e-0a1f4c69da0b admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:56.987942 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1145b1-bbed-47fd-9b5e-0a1f4c69da0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:56.987942 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1145b1-bbed-47fd-9b5e-0a1f4c69da0b admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:57.064509 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-f5598aad-3fc4-4f07-be6f-aec74fa108ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1dddc20f605b461a897a661942f381d7'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:57.084554 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-f5598aad-3fc4-4f07-be6f-aec74fa108ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1dddc20f605b461a897a661942f381d7'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:57.087333 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-f5598aad-3fc4-4f07-be6f-aec74fa108ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1dddc20f605b461a897a661942f381d7'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:57.154453 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 839/1656] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:54:56 2022] POST /identity/v3/projects => generated 366 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:54:57.158118 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1145b1-bbed-47fd-9b5e-0a1f4c69da0b admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:57.203327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-f5598aad-3fc4-4f07-be6f-aec74fa108ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1dddc20f605b461a897a661942f381d7'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:57.230553 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:54:57.233565 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-f5598aad-3fc4-4f07-be6f-aec74fa108ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1dddc20f605b461a897a661942f381d7'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:57.258344 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 818/1657] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:54:56 2022] DELETE /identity/v3/users/1dddc20f605b461a897a661942f381d7 => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:54:57.259609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5598aad-3fc4-4f07-be6f-aec74fa108ed admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:57.266144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f5598aad-3fc4-4f07-be6f-aec74fa108ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:57.346614 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1145b1-bbed-47fd-9b5e-0a1f4c69da0b admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:57.469377 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5598aad-3fc4-4f07-be6f-aec74fa108ed admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:57.509197 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd1d2d2-82fe-4fe1-98db-cdc711eb73eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:57.510038 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1d2d2-82fe-4fe1-98db-cdc711eb73eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:57.510434 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1d2d2-82fe-4fe1-98db-cdc711eb73eb admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:57.510826 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd1d2d2-82fe-4fe1-98db-cdc711eb73eb admin admin] PATH_INFO: `/v3/projects/9ac21ce7806d435991ede2da815318ee` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:57.518196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1d2d2-82fe-4fe1-98db-cdc711eb73eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ac21ce7806d435991ede2da815318ee)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:57.518196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: WARNING py.warnings [None req-fdd1d2d2-82fe-4fe1-98db-cdc711eb73eb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:54:57.518196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: warnings.warn(msg) Aug 02 10:54:57.518196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: Aug 02 10:54:57.518196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd1d2d2-82fe-4fe1-98db-cdc711eb73eb admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:57.567341 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca3e84-144d-4991-93df-36679fd4421e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:57.568039 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca3e84-144d-4991-93df-36679fd4421e admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:57.568369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca3e84-144d-4991-93df-36679fd4421e admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:57.568555 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca3e84-144d-4991-93df-36679fd4421e admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:57.569325 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca3e84-144d-4991-93df-36679fd4421e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:57.571329 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca3e84-144d-4991-93df-36679fd4421e admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:57.708207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 840/1658] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:54:57 2022] POST /identity/v3/users => generated 284 bytes in 551 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:54:57.712673 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a27ecd-d916-47e4-9468-3293b75cf958 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:57.712962 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a27ecd-d916-47e4-9468-3293b75cf958 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:57.713239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a27ecd-d916-47e4-9468-3293b75cf958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:57.771134 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-17a27ecd-d916-47e4-9468-3293b75cf958 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:57.772758 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-17a27ecd-d916-47e4-9468-3293b75cf958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:57.796361 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-fdd1d2d2-82fe-4fe1-98db-cdc711eb73eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ac21ce7806d435991ede2da815318ee', 'request_id': 'req-fdd1d2d2-82fe-4fe1-98db-cdc711eb73eb'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:57.894122 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-fdd1d2d2-82fe-4fe1-98db-cdc711eb73eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ac21ce7806d435991ede2da815318ee', 'request_id': 'req-fdd1d2d2-82fe-4fe1-98db-cdc711eb73eb'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:57.902295 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-fdd1d2d2-82fe-4fe1-98db-cdc711eb73eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ac21ce7806d435991ede2da815318ee', 'request_id': 'req-fdd1d2d2-82fe-4fe1-98db-cdc711eb73eb'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:57.917753 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 841/1659] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:57 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:57.918688 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a27ecd-d916-47e4-9468-3293b75cf958 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:57.920023 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-17a27ecd-d916-47e4-9468-3293b75cf958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:57.955109 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a27ecd-d916-47e4-9468-3293b75cf958 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:57.960885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-fdd1d2d2-82fe-4fe1-98db-cdc711eb73eb admin admin] The token cache is being invalidate because project 9ac21ce7806d435991ede2da815318ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:54:57.962395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-fdd1d2d2-82fe-4fe1-98db-cdc711eb73eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:57.963300 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-fdd1d2d2-82fe-4fe1-98db-cdc711eb73eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:57.965634 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 819/1660] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:54:57 2022] DELETE /identity/v3/projects/9ac21ce7806d435991ede2da815318ee => generated 0 bytes in 707 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:54:57.967862 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8399d6-8a93-486f-902f-eec9ef4fd8b2 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:57.968594 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8399d6-8a93-486f-902f-eec9ef4fd8b2 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:57.969577 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8399d6-8a93-486f-902f-eec9ef4fd8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:57.995185 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda1218e-22c0-4024-8630-9e325d30f475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:57.996761 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1218e-22c0-4024-8630-9e325d30f475 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:57.997852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1218e-22c0-4024-8630-9e325d30f475 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:58.001869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda1218e-22c0-4024-8630-9e325d30f475 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:58.006147 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda1218e-22c0-4024-8630-9e325d30f475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:58.011359 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-7d8399d6-8a93-486f-902f-eec9ef4fd8b2 None None] MFA Rules not processed for user `fbf5248a340944bb831f446f97426409`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:58.012366 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda1218e-22c0-4024-8630-9e325d30f475 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:58.013498 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-7d8399d6-8a93-486f-902f-eec9ef4fd8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:58.025093 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:54:58.067842 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 842/1661] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:54:57 2022] GET /identity/v3/roles => generated 1427 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:54:58.072179 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117a250-5770-45ba-af4d-1805063eb81a None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:58.073005 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117a250-5770-45ba-af4d-1805063eb81a None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:58.075497 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d117a250-5770-45ba-af4d-1805063eb81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:58.125735 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 820/1662] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:57 2022] POST /identity/v3/auth/tokens => generated 3357 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:58.126390 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8399d6-8a93-486f-902f-eec9ef4fd8b2 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:58.141042 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-7d8399d6-8a93-486f-902f-eec9ef4fd8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:58.159128 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:54:58.225125 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8399d6-8a93-486f-902f-eec9ef4fd8b2 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:58.230421 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-d117a250-5770-45ba-af4d-1805063eb81a None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:58.231736 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ced55-e128-4698-a072-7547270cd6e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:58.232380 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ced55-e128-4698-a072-7547270cd6e8 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:58.232656 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ced55-e128-4698-a072-7547270cd6e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:58.232928 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ced55-e128-4698-a072-7547270cd6e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:58.236055 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-e96ced55-e128-4698-a072-7547270cd6e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:58.238302 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-d117a250-5770-45ba-af4d-1805063eb81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:58.268856 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ced55-e128-4698-a072-7547270cd6e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:58.270783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ced55-e128-4698-a072-7547270cd6e8 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:58.283895 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 843/1663] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:58 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:58.287725 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d117a250-5770-45ba-af4d-1805063eb81a None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:58.293405 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 821/1664] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:54:58 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:54:58.294809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d194c-5377-48c0-b840-d33073547332 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:58.295226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d194c-5377-48c0-b840-d33073547332 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:58.303311 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d194c-5377-48c0-b840-d33073547332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:58.303474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-d117a250-5770-45ba-af4d-1805063eb81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:58.334271 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-ce3d194c-5377-48c0-b840-d33073547332 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:58.335738 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-ce3d194c-5377-48c0-b840-d33073547332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:58.347173 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 822/1665] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:58 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:58.352104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d194c-5377-48c0-b840-d33073547332 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:58.364767 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d117a250-5770-45ba-af4d-1805063eb81a None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:58.368852 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d194c-5377-48c0-b840-d33073547332 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:58.375395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39705e43-ed00-497f-8eaf-c1537da23ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:58.376308 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39705e43-ed00-497f-8eaf-c1537da23ad9 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:58.376650 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39705e43-ed00-497f-8eaf-c1537da23ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:58.377283 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39705e43-ed00-497f-8eaf-c1537da23ad9 admin admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:58.378551 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39705e43-ed00-497f-8eaf-c1537da23ad9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:54:58.378953 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39705e43-ed00-497f-8eaf-c1537da23ad9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:58.380760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39705e43-ed00-497f-8eaf-c1537da23ad9 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:58.386293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aadf4fa-b6cc-41b7-b8c8-02cd04ab97b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:58.387989 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aadf4fa-b6cc-41b7-b8c8-02cd04ab97b8 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:58.388646 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aadf4fa-b6cc-41b7-b8c8-02cd04ab97b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:58.392162 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aadf4fa-b6cc-41b7-b8c8-02cd04ab97b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:58.393465 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aadf4fa-b6cc-41b7-b8c8-02cd04ab97b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:58.396265 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 823/1666] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:54:58 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:54:58.400096 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aadf4fa-b6cc-41b7-b8c8-02cd04ab97b8 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:58.400985 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39705e43-ed00-497f-8eaf-c1537da23ad9 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:58.413755 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39705e43-ed00-497f-8eaf-c1537da23ad9 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:58.421532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f10247-053f-414d-add1-62217e816cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:58.425549 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f10247-053f-414d-add1-62217e816cb2 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:58.425549 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f10247-053f-414d-add1-62217e816cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:58.425549 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f10247-053f-414d-add1-62217e816cb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:58.425549 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f10247-053f-414d-add1-62217e816cb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:58.427978 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f10247-053f-414d-add1-62217e816cb2 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:58.446506 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 844/1667] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:54:58 2022] GET /identity/v3/roles => generated 1427 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:54:58.457639 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aadf4fa-b6cc-41b7-b8c8-02cd04ab97b8 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:58.477548 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aadf4fa-b6cc-41b7-b8c8-02cd04ab97b8 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:58.495366 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc0f57b-2247-4cc1-bc1a-61009d62962b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:58.496354 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc0f57b-2247-4cc1-bc1a-61009d62962b admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:58.496801 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc0f57b-2247-4cc1-bc1a-61009d62962b admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:58.497250 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc0f57b-2247-4cc1-bc1a-61009d62962b admin admin] PATH_INFO: `/v3/projects/fb73b626b16c4764b9f7336c0840aae0/users/36e060a0d20749dd85fa18f081d67192/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:58.538394 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 824/1668] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:54:58 2022] POST /identity/v3/projects => generated 388 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:54:58.538788 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc0f57b-2247-4cc1-bc1a-61009d62962b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb73b626b16c4764b9f7336c0840aae0, user_id=36e060a0d20749dd85fa18f081d67192, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:58.542260 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f10247-053f-414d-add1-62217e816cb2 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:58.543084 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc0f57b-2247-4cc1-bc1a-61009d62962b admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:58.662956 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 845/1669] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:54:58 2022] PUT /identity/v3/projects/fb73b626b16c4764b9f7336c0840aae0/users/36e060a0d20749dd85fa18f081d67192/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:54:58.665014 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f2834-fa9d-4bd8-9d59-425b0a0b0333 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:58.665413 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f2834-fa9d-4bd8-9d59-425b0a0b0333 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:58.668953 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117f2834-fa9d-4bd8-9d59-425b0a0b0333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:58.729630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f10247-053f-414d-add1-62217e816cb2 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:58.756560 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-117f2834-fa9d-4bd8-9d59-425b0a0b0333 None None] MFA Rules not processed for user `36e060a0d20749dd85fa18f081d67192`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:58.756560 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-117f2834-fa9d-4bd8-9d59-425b0a0b0333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:58.894660 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeaee6b-b011-4596-a0d9-334d35ae9585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:58.900895 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeaee6b-b011-4596-a0d9-334d35ae9585 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:58.904640 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeaee6b-b011-4596-a0d9-334d35ae9585 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:58.904979 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeaee6b-b011-4596-a0d9-334d35ae9585 admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:58.905729 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeaee6b-b011-4596-a0d9-334d35ae9585 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:58.907863 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeaee6b-b011-4596-a0d9-334d35ae9585 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:58.934882 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 846/1670] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:58 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:54:58.940379 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f2834-fa9d-4bd8-9d59-425b0a0b0333 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:58.963236 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117f2834-fa9d-4bd8-9d59-425b0a0b0333 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:58.979290 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd4b8e-2b65-41b3-b9cf-de3bd01f57a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:58.979799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd4b8e-2b65-41b3-b9cf-de3bd01f57a0 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:58.980081 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd4b8e-2b65-41b3-b9cf-de3bd01f57a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:58.980355 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd4b8e-2b65-41b3-b9cf-de3bd01f57a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:58.981046 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd4b8e-2b65-41b3-b9cf-de3bd01f57a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:58.982981 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edd4b8e-2b65-41b3-b9cf-de3bd01f57a0 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:59.060204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 825/1671] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:54:58 2022] POST /identity/v3/users => generated 295 bytes in 518 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:54:59.065127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeaee6b-b011-4596-a0d9-334d35ae9585 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:59.083580 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeaee6b-b011-4596-a0d9-334d35ae9585 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:59.097775 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7913836-46e7-4b08-b1f5-be519669726f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:59.098691 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7913836-46e7-4b08-b1f5-be519669726f admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:59.098846 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7913836-46e7-4b08-b1f5-be519669726f admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:59.099146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7913836-46e7-4b08-b1f5-be519669726f admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:59.099919 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7913836-46e7-4b08-b1f5-be519669726f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:59.101530 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7913836-46e7-4b08-b1f5-be519669726f admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:59.148902 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 847/1672] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:54:58 2022] POST /identity/v3/projects => generated 364 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:54:59.149991 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd4b8e-2b65-41b3-b9cf-de3bd01f57a0 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:59.151103 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 826/1673] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:54:59 2022] GET /identity/v3/roles => generated 1427 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:54:59.154645 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7913836-46e7-4b08-b1f5-be519669726f admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:59.180820 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7913836-46e7-4b08-b1f5-be519669726f admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:59.317765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd4b8e-2b65-41b3-b9cf-de3bd01f57a0 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:59.336447 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122fb547-905c-4c9e-85f2-84b3fbcbafcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:59.338243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fb547-905c-4c9e-85f2-84b3fbcbafcb admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:59.338539 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fb547-905c-4c9e-85f2-84b3fbcbafcb admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:59.339824 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122fb547-905c-4c9e-85f2-84b3fbcbafcb admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:59.344091 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122fb547-905c-4c9e-85f2-84b3fbcbafcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:59.345179 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c033c7b-55d6-48d5-b413-b400caf0677a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:59.345852 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c033c7b-55d6-48d5-b413-b400caf0677a admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:59.349352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c033c7b-55d6-48d5-b413-b400caf0677a admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:59.349642 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c033c7b-55d6-48d5-b413-b400caf0677a admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:59.350178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122fb547-905c-4c9e-85f2-84b3fbcbafcb admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:59.352187 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c033c7b-55d6-48d5-b413-b400caf0677a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:59.364051 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c033c7b-55d6-48d5-b413-b400caf0677a admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:59.375765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 848/1674] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:54:59 2022] GET /identity/v3/roles => generated 1427 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:54:59.376920 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122fb547-905c-4c9e-85f2-84b3fbcbafcb admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:59.396296 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122fb547-905c-4c9e-85f2-84b3fbcbafcb admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:59.403912 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2601c712-8a92-472c-88f6-a82bc1a5f23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:59.405149 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601c712-8a92-472c-88f6-a82bc1a5f23d admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:59.406184 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601c712-8a92-472c-88f6-a82bc1a5f23d admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:59.406375 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2601c712-8a92-472c-88f6-a82bc1a5f23d admin admin] PATH_INFO: `/v3/projects/d709dc6793fb484ea9e293e491c75727/users/02795c3a9f9e451281b955bade62d9ac/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:59.432891 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2601c712-8a92-472c-88f6-a82bc1a5f23d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d709dc6793fb484ea9e293e491c75727, user_id=02795c3a9f9e451281b955bade62d9ac, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:59.435978 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2601c712-8a92-472c-88f6-a82bc1a5f23d admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:59.440767 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:54:59.502122 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 827/1675] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:54:59 2022] POST /identity/v3/users => generated 289 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:54:59.507119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c033c7b-55d6-48d5-b413-b400caf0677a admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:59.512956 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-0c033c7b-55d6-48d5-b413-b400caf0677a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:59.553253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c033c7b-55d6-48d5-b413-b400caf0677a admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:59.556070 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:54:59.570370 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 849/1676] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:54:59 2022] PUT /identity/v3/projects/d709dc6793fb484ea9e293e491c75727/users/02795c3a9f9e451281b955bade62d9ac/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:54:59.571641 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2601c712-8a92-472c-88f6-a82bc1a5f23d admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:54:59.704188 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bdaa60-396d-4477-91b5-7a8ed7d55a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:59.709163 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bdaa60-396d-4477-91b5-7a8ed7d55a19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:59.709163 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bdaa60-396d-4477-91b5-7a8ed7d55a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:59.709163 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bdaa60-396d-4477-91b5-7a8ed7d55a19 admin admin] PATH_INFO: `/v3/projects/9d121c6b6fc44c4ca99caf69e4f57436` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:59.710100 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bdaa60-396d-4477-91b5-7a8ed7d55a19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d121c6b6fc44c4ca99caf69e4f57436)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:59.716089 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bdaa60-396d-4477-91b5-7a8ed7d55a19 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:59.718911 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2601c712-8a92-472c-88f6-a82bc1a5f23d admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:54:59.738421 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a87091-d2bd-4dcd-aee7-bc10123d1d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:54:59.739036 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a87091-d2bd-4dcd-aee7-bc10123d1d16 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:59.739344 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a87091-d2bd-4dcd-aee7-bc10123d1d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:59.739679 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a87091-d2bd-4dcd-aee7-bc10123d1d16 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:59.740355 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a87091-d2bd-4dcd-aee7-bc10123d1d16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:54:59.742003 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a87091-d2bd-4dcd-aee7-bc10123d1d16 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:54:59.764387 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 850/1677] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:54:59 2022] GET /identity/v3/roles => generated 1427 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:54:59.764387 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd53f7d-3932-4b5b-8432-b6d333af7c2c None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:54:59.764387 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd53f7d-3932-4b5b-8432-b6d333af7c2c None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:54:59.764387 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd53f7d-3932-4b5b-8432-b6d333af7c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:54:59.837328 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-abd53f7d-3932-4b5b-8432-b6d333af7c2c None None] MFA Rules not processed for user `02795c3a9f9e451281b955bade62d9ac`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:54:59.838587 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-abd53f7d-3932-4b5b-8432-b6d333af7c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:54:59.943827 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-30bdaa60-396d-4477-91b5-7a8ed7d55a19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d121c6b6fc44c4ca99caf69e4f57436', 'request_id': 'req-30bdaa60-396d-4477-91b5-7a8ed7d55a19'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:54:59.995748 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 851/1678] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:54:59 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:00.001129 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd53f7d-3932-4b5b-8432-b6d333af7c2c None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:00.018724 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-30bdaa60-396d-4477-91b5-7a8ed7d55a19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d121c6b6fc44c4ca99caf69e4f57436', 'request_id': 'req-30bdaa60-396d-4477-91b5-7a8ed7d55a19'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:00.027365 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-30bdaa60-396d-4477-91b5-7a8ed7d55a19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d121c6b6fc44c4ca99caf69e4f57436', 'request_id': 'req-30bdaa60-396d-4477-91b5-7a8ed7d55a19'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:00.030289 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd53f7d-3932-4b5b-8432-b6d333af7c2c None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:00.083687 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-30bdaa60-396d-4477-91b5-7a8ed7d55a19 admin admin] The token cache is being invalidate because project 9d121c6b6fc44c4ca99caf69e4f57436 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:55:00.083687 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-30bdaa60-396d-4477-91b5-7a8ed7d55a19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:00.086635 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-30bdaa60-396d-4477-91b5-7a8ed7d55a19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:00.096191 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 828/1679] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:54:59 2022] DELETE /identity/v3/projects/9d121c6b6fc44c4ca99caf69e4f57436 => generated 0 bytes in 590 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:00.097629 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bdaa60-396d-4477-91b5-7a8ed7d55a19 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:00.109225 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-30bdaa60-396d-4477-91b5-7a8ed7d55a19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:00.183920 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24747429-ceb0-41a3-a9b9-777e3473d323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:00.184642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24747429-ceb0-41a3-a9b9-777e3473d323 admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:00.184928 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24747429-ceb0-41a3-a9b9-777e3473d323 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:00.185204 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24747429-ceb0-41a3-a9b9-777e3473d323 admin admin] PATH_INFO: `/v3/projects/7a05dc80ed4a4c4b87c1e5c96340a488/users/042a99b0f6ae45c1b61c06256284f32b/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:00.218780 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24747429-ceb0-41a3-a9b9-777e3473d323 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a05dc80ed4a4c4b87c1e5c96340a488, user_id=042a99b0f6ae45c1b61c06256284f32b, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:00.222111 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24747429-ceb0-41a3-a9b9-777e3473d323 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:00.244664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bdaa60-396d-4477-91b5-7a8ed7d55a19 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:00.254213 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bdfeae-2a8e-4b09-900a-bf2d309525cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:00.254890 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bdfeae-2a8e-4b09-900a-bf2d309525cd admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:00.255199 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bdfeae-2a8e-4b09-900a-bf2d309525cd admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:00.255476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bdfeae-2a8e-4b09-900a-bf2d309525cd admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:00.256220 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bdfeae-2a8e-4b09-900a-bf2d309525cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:00.259799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bdfeae-2a8e-4b09-900a-bf2d309525cd admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:00.307015 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 852/1680] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:54:59 2022] PUT /identity/v3/projects/7a05dc80ed4a4c4b87c1e5c96340a488/users/042a99b0f6ae45c1b61c06256284f32b/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:00.318123 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d155e7-2e5d-4ce2-8ba2-d0e3264961dc None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:00.318123 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d155e7-2e5d-4ce2-8ba2-d0e3264961dc None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:00.318123 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d155e7-2e5d-4ce2-8ba2-d0e3264961dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:00.331079 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:00.354190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-b3d155e7-2e5d-4ce2-8ba2-d0e3264961dc None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:00.355848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-b3d155e7-2e5d-4ce2-8ba2-d0e3264961dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:00.392399 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 829/1681] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:55:00 2022] POST /identity/v3/projects => generated 386 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:00.393372 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bdfeae-2a8e-4b09-900a-bf2d309525cd admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:00.399135 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-04bdfeae-2a8e-4b09-900a-bf2d309525cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:00.461841 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:00.518522 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 853/1682] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:00 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:00.518522 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d155e7-2e5d-4ce2-8ba2-d0e3264961dc None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:00.535594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d155e7-2e5d-4ce2-8ba2-d0e3264961dc None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:00.593687 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bdfeae-2a8e-4b09-900a-bf2d309525cd admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:00.606849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c835ac0-6f47-4bc1-8d9c-6a5c1f1a364f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:00.608976 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c835ac0-6f47-4bc1-8d9c-6a5c1f1a364f admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:00.609364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c835ac0-6f47-4bc1-8d9c-6a5c1f1a364f admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:00.609650 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c835ac0-6f47-4bc1-8d9c-6a5c1f1a364f admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:00.612472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c835ac0-6f47-4bc1-8d9c-6a5c1f1a364f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:00.618050 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c835ac0-6f47-4bc1-8d9c-6a5c1f1a364f admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:00.647135 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 830/1683] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:00 2022] GET /identity/v3/roles => generated 1427 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:00.649347 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c62e9-b26e-48e9-81f1-5c2c660ee373 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:00.649643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c62e9-b26e-48e9-81f1-5c2c660ee373 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:00.649923 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0c62e9-b26e-48e9-81f1-5c2c660ee373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:00.678236 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4800d5fe-b8e2-4194-a137-34e117a67cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:00.680408 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800d5fe-b8e2-4194-a137-34e117a67cb7 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:00.680802 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800d5fe-b8e2-4194-a137-34e117a67cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:00.681225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800d5fe-b8e2-4194-a137-34e117a67cb7 admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:00.682191 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4800d5fe-b8e2-4194-a137-34e117a67cb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:00.684928 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4800d5fe-b8e2-4194-a137-34e117a67cb7 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:00.722273 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-1c0c62e9-b26e-48e9-81f1-5c2c660ee373 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:00.729222 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-1c0c62e9-b26e-48e9-81f1-5c2c660ee373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:00.766267 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 831/1684] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:00 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:00.767997 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c62e9-b26e-48e9-81f1-5c2c660ee373 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:00.799639 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 854/1685] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:55:00 2022] POST /identity/v3/users => generated 294 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:00.800722 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4800d5fe-b8e2-4194-a137-34e117a67cb7 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:00.812858 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0c62e9-b26e-48e9-81f1-5c2c660ee373 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:00.826713 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60f843-a19b-496a-b775-4be5e3055b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:00.830429 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60f843-a19b-496a-b775-4be5e3055b68 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:00.830776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60f843-a19b-496a-b775-4be5e3055b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:00.832278 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60f843-a19b-496a-b775-4be5e3055b68 admin admin] PATH_INFO: `/v3/projects/7a05dc80ed4a4c4b87c1e5c96340a488/users/042a99b0f6ae45c1b61c06256284f32b/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:00.839905 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf60f843-a19b-496a-b775-4be5e3055b68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a05dc80ed4a4c4b87c1e5c96340a488, user_id=042a99b0f6ae45c1b61c06256284f32b, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:00.842460 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4800d5fe-b8e2-4194-a137-34e117a67cb7 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:00.848069 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf60f843-a19b-496a-b775-4be5e3055b68 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:00.853593 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b065e316-8bb9-41e7-a603-97550a8d2197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:00.863333 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065e316-8bb9-41e7-a603-97550a8d2197 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:00.864597 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065e316-8bb9-41e7-a603-97550a8d2197 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:00.865069 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b065e316-8bb9-41e7-a603-97550a8d2197 admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:00.871148 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065e316-8bb9-41e7-a603-97550a8d2197 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:55:00.871148 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065e316-8bb9-41e7-a603-97550a8d2197 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:00.871148 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b065e316-8bb9-41e7-a603-97550a8d2197 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:00.883092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 832/1686] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:00 2022] PUT /identity/v3/projects/7a05dc80ed4a4c4b87c1e5c96340a488/users/042a99b0f6ae45c1b61c06256284f32b/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:00.883092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60f843-a19b-496a-b775-4be5e3055b68 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:00.916703 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 855/1687] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:55:00 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:55:00.918273 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acefa35-46b7-445f-a6a6-ad2b7a023cd3 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:00.920992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acefa35-46b7-445f-a6a6-ad2b7a023cd3 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:00.922164 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acefa35-46b7-445f-a6a6-ad2b7a023cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:00.988525 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-2acefa35-46b7-445f-a6a6-ad2b7a023cd3 None None] MFA Rules not processed for user `042a99b0f6ae45c1b61c06256284f32b`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:00.991326 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-2acefa35-46b7-445f-a6a6-ad2b7a023cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:01.082772 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60f843-a19b-496a-b775-4be5e3055b68 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:01.098737 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac6d61-7567-42fc-95b9-a20d83c32e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:01.099678 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac6d61-7567-42fc-95b9-a20d83c32e8a admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:01.100144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac6d61-7567-42fc-95b9-a20d83c32e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:01.100588 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fac6d61-7567-42fc-95b9-a20d83c32e8a admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:01.101507 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fac6d61-7567-42fc-95b9-a20d83c32e8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:01.103882 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fac6d61-7567-42fc-95b9-a20d83c32e8a admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:01.117850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 833/1688] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:00 2022] GET /identity/v3/roles => generated 1427 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:01.118720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac6d61-7567-42fc-95b9-a20d83c32e8a admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:01.131687 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fac6d61-7567-42fc-95b9-a20d83c32e8a admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:01.137947 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23097b94-5443-416b-8d1b-7f64d48588a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:01.140884 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23097b94-5443-416b-8d1b-7f64d48588a7 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:01.141417 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23097b94-5443-416b-8d1b-7f64d48588a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:01.141829 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23097b94-5443-416b-8d1b-7f64d48588a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:01.142631 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23097b94-5443-416b-8d1b-7f64d48588a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:01.145736 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23097b94-5443-416b-8d1b-7f64d48588a7 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:01.196093 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 856/1689] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:00 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:01.198007 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acefa35-46b7-445f-a6a6-ad2b7a023cd3 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:01.214805 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acefa35-46b7-445f-a6a6-ad2b7a023cd3 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:01.220903 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517a39d-8a05-43d9-bddc-b87cd459715c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:01.221551 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517a39d-8a05-43d9-bddc-b87cd459715c admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:01.221835 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517a39d-8a05-43d9-bddc-b87cd459715c admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:01.222128 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517a39d-8a05-43d9-bddc-b87cd459715c admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:01.222832 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8517a39d-8a05-43d9-bddc-b87cd459715c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:01.233220 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8517a39d-8a05-43d9-bddc-b87cd459715c admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:01.235853 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:01.237933 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 834/1690] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:55:01 2022] POST /identity/v3/projects => generated 368 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:01.239119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23097b94-5443-416b-8d1b-7f64d48588a7 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:01.243456 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-23097b94-5443-416b-8d1b-7f64d48588a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:01.280930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 857/1691] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:01 2022] GET /identity/v3/roles => generated 1427 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:01.281992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517a39d-8a05-43d9-bddc-b87cd459715c admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:01.318960 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517a39d-8a05-43d9-bddc-b87cd459715c admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:01.365430 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:01.461034 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23097b94-5443-416b-8d1b-7f64d48588a7 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:01.473817 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985fe6f3-34df-418d-80b5-f258d39e8cda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:01.474469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fe6f3-34df-418d-80b5-f258d39e8cda service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:01.474752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fe6f3-34df-418d-80b5-f258d39e8cda service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:01.476496 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fe6f3-34df-418d-80b5-f258d39e8cda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:01.515005 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2dac24-3789-465c-8d11-604b16113637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:01.519219 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2dac24-3789-465c-8d11-604b16113637 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:01.519511 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2dac24-3789-465c-8d11-604b16113637 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:01.519792 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2dac24-3789-465c-8d11-604b16113637 admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:01.520485 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2dac24-3789-465c-8d11-604b16113637 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:01.528308 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2dac24-3789-465c-8d11-604b16113637 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:01.650997 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985fe6f3-34df-418d-80b5-f258d39e8cda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:01.650997 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985fe6f3-34df-418d-80b5-f258d39e8cda service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:01.668733 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 835/1692] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:01 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:01.669669 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985fe6f3-34df-418d-80b5-f258d39e8cda service cinder] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:01.683847 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985fe6f3-34df-418d-80b5-f258d39e8cda service cinder] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:01.716582 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 858/1693] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:55:01 2022] POST /identity/v3/users => generated 285 bytes in 434 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:01.762023 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34602abe-93a8-43b6-a87f-050c7411f377 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:01.762023 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34602abe-93a8-43b6-a87f-050c7411f377 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:01.762023 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34602abe-93a8-43b6-a87f-050c7411f377 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:01.762653 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a246ba-d21f-4dea-9010-cf0b49e0cdfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:01.762653 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a246ba-d21f-4dea-9010-cf0b49e0cdfc admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:01.762653 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a246ba-d21f-4dea-9010-cf0b49e0cdfc admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:01.762653 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a246ba-d21f-4dea-9010-cf0b49e0cdfc admin admin] PATH_INFO: `/v3/projects/f4a82fa22d1f4773a0bafe9a0ef7e951/users/bdf2d127118f4d78979c4935da3d1756/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:01.762987 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 859/1694] 10.222.0.115 () {54 vars in 926 bytes} [Tue Aug 2 10:55:01 2022] GET /identity => generated 271 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:55:01.763108 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a246ba-d21f-4dea-9010-cf0b49e0cdfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4a82fa22d1f4773a0bafe9a0ef7e951, user_id=bdf2d127118f4d78979c4935da3d1756, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:01.763108 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a246ba-d21f-4dea-9010-cf0b49e0cdfc admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:01.763298 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34602abe-93a8-43b6-a87f-050c7411f377 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:01.788616 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34602abe-93a8-43b6-a87f-050c7411f377 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:01.794283 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83c8b9-f0ca-4540-90d0-739ee152e10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:01.795228 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83c8b9-f0ca-4540-90d0-739ee152e10f admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:01.795663 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83c8b9-f0ca-4540-90d0-739ee152e10f admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:01.796633 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b83c8b9-f0ca-4540-90d0-739ee152e10f admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:01.797518 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b83c8b9-f0ca-4540-90d0-739ee152e10f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:01.799428 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b83c8b9-f0ca-4540-90d0-739ee152e10f admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:01.818469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 860/1695] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:01 2022] GET /identity/v3/roles => generated 1427 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:55:01.823435 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83c8b9-f0ca-4540-90d0-739ee152e10f admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:01.846443 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b83c8b9-f0ca-4540-90d0-739ee152e10f admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:01.855698 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c597f7-e27b-4187-9851-9e8d03672019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:01.856658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c597f7-e27b-4187-9851-9e8d03672019 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:01.857110 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c597f7-e27b-4187-9851-9e8d03672019 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:01.857541 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c597f7-e27b-4187-9851-9e8d03672019 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:01.859409 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c597f7-e27b-4187-9851-9e8d03672019 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:01.861656 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c597f7-e27b-4187-9851-9e8d03672019 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:01.883304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 861/1696] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:01 2022] GET /identity/v3/roles => generated 1427 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:01.885024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ee584-a3f3-46c5-ba64-caac372a3ea9 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:01.885951 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ee584-a3f3-46c5-ba64-caac372a3ea9 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:01.886408 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ee584-a3f3-46c5-ba64-caac372a3ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:01.919670 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-851ee584-a3f3-46c5-ba64-caac372a3ea9 None None] MFA Rules not processed for user `cbc4b962e2a742b08beef7187cc527e5`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:01.922142 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-851ee584-a3f3-46c5-ba64-caac372a3ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:01.932069 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 836/1697] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:01 2022] PUT /identity/v3/projects/f4a82fa22d1f4773a0bafe9a0ef7e951/users/bdf2d127118f4d78979c4935da3d1756/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:01.936171 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a246ba-d21f-4dea-9010-cf0b49e0cdfc admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:02.115286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a246ba-d21f-4dea-9010-cf0b49e0cdfc admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:02.130066 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 862/1698] 10.222.0.115 () {58 vars in 1023 bytes} [Tue Aug 2 10:55:01 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:02.144889 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fdd67e-920d-4cac-b029-9bb0c46fffdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:02.145327 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23ebcf-2d69-4ad5-be4d-59106376c235 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:02.145888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fdd67e-920d-4cac-b029-9bb0c46fffdb admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:02.146055 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23ebcf-2d69-4ad5-be4d-59106376c235 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:02.146384 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d23ebcf-2d69-4ad5-be4d-59106376c235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:02.156766 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fdd67e-920d-4cac-b029-9bb0c46fffdb admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:02.159113 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fdd67e-920d-4cac-b029-9bb0c46fffdb admin admin] PATH_INFO: `/v3/projects/bc5aa3b9a1e24cf4bf525f850525f624/users/31bc36c4e2874704bb06a571b6073873/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:02.222570 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fdd67e-920d-4cac-b029-9bb0c46fffdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc5aa3b9a1e24cf4bf525f850525f624, user_id=31bc36c4e2874704bb06a571b6073873, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:02.226030 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fdd67e-920d-4cac-b029-9bb0c46fffdb admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:02.260974 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-4d23ebcf-2d69-4ad5-be4d-59106376c235 None None] MFA Rules not processed for user `bdf2d127118f4d78979c4935da3d1756`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:02.262835 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-4d23ebcf-2d69-4ad5-be4d-59106376c235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:02.314781 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 837/1699] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:01 2022] PUT /identity/v3/projects/bc5aa3b9a1e24cf4bf525f850525f624/users/31bc36c4e2874704bb06a571b6073873/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:02.319727 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fdd67e-920d-4cac-b029-9bb0c46fffdb admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:02.334226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:02.409658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:02.463148 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 863/1700] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:02 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:02.465689 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d340a4-0448-4f8c-90aa-d6b36fc98ed0 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:02.465689 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d340a4-0448-4f8c-90aa-d6b36fc98ed0 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:02.465689 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d340a4-0448-4f8c-90aa-d6b36fc98ed0 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:02.474078 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 864/1701] 10.222.0.115 () {54 vars in 966 bytes} [Tue Aug 2 10:55:02 2022] GET /identity => generated 271 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:55:02.475497 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f7f6a-f20a-47d3-a012-624c4d87a6f1 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:02.475787 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f7f6a-f20a-47d3-a012-624c4d87a6f1 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:02.476054 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f7f6a-f20a-47d3-a012-624c4d87a6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:02.486097 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fdd67e-920d-4cac-b029-9bb0c46fffdb admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:02.491851 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31161e-568d-485f-aa1a-eb0068e6eae7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:02.493019 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31161e-568d-485f-aa1a-eb0068e6eae7 service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:02.493516 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31161e-568d-485f-aa1a-eb0068e6eae7 service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:02.494008 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b31161e-568d-485f-aa1a-eb0068e6eae7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:02.496315 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-1b31161e-568d-485f-aa1a-eb0068e6eae7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:02.580830 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-d50f7f6a-f20a-47d3-a012-624c4d87a6f1 None None] MFA Rules not processed for user `31bc36c4e2874704bb06a571b6073873`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:02.588969 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-d50f7f6a-f20a-47d3-a012-624c4d87a6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:02.595984 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31161e-568d-485f-aa1a-eb0068e6eae7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:02.597300 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b31161e-568d-485f-aa1a-eb0068e6eae7 service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:02.614752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 838/1702] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:02.614752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31161e-568d-485f-aa1a-eb0068e6eae7 service cinder] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:02.733666 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b31161e-568d-485f-aa1a-eb0068e6eae7 service cinder] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:02.753683 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71027e-8f99-4c8e-8a60-f7de6946822d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:02.755612 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71027e-8f99-4c8e-8a60-f7de6946822d admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:02.756102 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71027e-8f99-4c8e-8a60-f7de6946822d admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:02.756688 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb71027e-8f99-4c8e-8a60-f7de6946822d admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:02.757800 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb71027e-8f99-4c8e-8a60-f7de6946822d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:02.760446 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb71027e-8f99-4c8e-8a60-f7de6946822d admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:02.815000 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 865/1703] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 340 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:02.816408 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b04bb-9e8a-4133-8b89-321fe08622f8 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:02.818132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b04bb-9e8a-4133-8b89-321fe08622f8 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:02.818132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b04bb-9e8a-4133-8b89-321fe08622f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:02.852471 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-b87b04bb-9e8a-4133-8b89-321fe08622f8 None None] MFA Rules not processed for user `cbc4b962e2a742b08beef7187cc527e5`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:02.853708 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-b87b04bb-9e8a-4133-8b89-321fe08622f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:02.887606 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 839/1704] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:55:02 2022] POST /identity/v3/projects => generated 388 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:02.888594 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71027e-8f99-4c8e-8a60-f7de6946822d admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:02.898246 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-eb71027e-8f99-4c8e-8a60-f7de6946822d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:03.049834 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 866/1705] 10.222.0.115 () {58 vars in 1063 bytes} [Tue Aug 2 10:55:02 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:03.052517 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b04bb-9e8a-4133-8b89-321fe08622f8 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:03.078603 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b04bb-9e8a-4133-8b89-321fe08622f8 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:03.096622 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:03.118089 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb71027e-8f99-4c8e-8a60-f7de6946822d admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:03.126335 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376df3b-570f-494d-ada3-db2d028f0e7e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:03.128953 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376df3b-570f-494d-ada3-db2d028f0e7e service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:03.135499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376df3b-570f-494d-ada3-db2d028f0e7e service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:03.135499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5376df3b-570f-494d-ada3-db2d028f0e7e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:03.249897 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da867b53-8f1d-4782-8ed6-1ae7c99066a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:03.251384 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da867b53-8f1d-4782-8ed6-1ae7c99066a6 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:03.251693 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da867b53-8f1d-4782-8ed6-1ae7c99066a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:03.251972 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da867b53-8f1d-4782-8ed6-1ae7c99066a6 admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:03.252660 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da867b53-8f1d-4782-8ed6-1ae7c99066a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:03.255025 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da867b53-8f1d-4782-8ed6-1ae7c99066a6 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:03.279673 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5376df3b-570f-494d-ada3-db2d028f0e7e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:03.281521 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5376df3b-570f-494d-ada3-db2d028f0e7e service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:03.312251 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 840/1706] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:03.317815 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376df3b-570f-494d-ada3-db2d028f0e7e service cinder] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:03.397586 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 867/1707] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:55:03 2022] POST /identity/v3/users => generated 301 bytes in 343 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:03.398899 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da867b53-8f1d-4782-8ed6-1ae7c99066a6 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:03.415908 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:03.430808 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da867b53-8f1d-4782-8ed6-1ae7c99066a6 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:03.450334 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f7a74-e2ec-4fff-8a21-641fee7fd331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:03.454283 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f7a74-e2ec-4fff-8a21-641fee7fd331 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:03.454790 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f7a74-e2ec-4fff-8a21-641fee7fd331 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:03.455238 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45f7a74-e2ec-4fff-8a21-641fee7fd331 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:03.459541 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45f7a74-e2ec-4fff-8a21-641fee7fd331 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:03.462970 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45f7a74-e2ec-4fff-8a21-641fee7fd331 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:03.486517 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 868/1708] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:03 2022] GET /identity/v3/roles => generated 1427 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:03.491501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f7a74-e2ec-4fff-8a21-641fee7fd331 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:03.506848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45f7a74-e2ec-4fff-8a21-641fee7fd331 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:03.519205 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59b276-c255-4178-b6b4-2ee10097cabd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:03.519205 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59b276-c255-4178-b6b4-2ee10097cabd admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:03.519205 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59b276-c255-4178-b6b4-2ee10097cabd admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:03.519205 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59b276-c255-4178-b6b4-2ee10097cabd admin admin] PATH_INFO: `/v3/projects/85e2e21340eb4754be5a6bbe2e56905a/users/78a9147f3daf4c64a9261c4297b3ff9c/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:03.547364 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d59b276-c255-4178-b6b4-2ee10097cabd admin admin] RBAC: Authorizing `identity:create_grant(project_id=85e2e21340eb4754be5a6bbe2e56905a, user_id=78a9147f3daf4c64a9261c4297b3ff9c, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:03.551944 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d59b276-c255-4178-b6b4-2ee10097cabd admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:03.562100 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5376df3b-570f-494d-ada3-db2d028f0e7e service cinder] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:03.573737 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d8259-97de-4ef1-8fa5-a1fd7093ce1d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc4b962e2a742b08beef7187cc527e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:03.583215 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d8259-97de-4ef1-8fa5-a1fd7093ce1d service placement] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:03.583215 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d8259-97de-4ef1-8fa5-a1fd7093ce1d service placement] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:03.583215 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912d8259-97de-4ef1-8fa5-a1fd7093ce1d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:03.588271 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d8259-97de-4ef1-8fa5-a1fd7093ce1d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:03.590302 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912d8259-97de-4ef1-8fa5-a1fd7093ce1d service placement] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:03.607206 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 841/1709] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:03 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:03.651757 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 869/1710] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:03 2022] PUT /identity/v3/projects/85e2e21340eb4754be5a6bbe2e56905a/users/78a9147f3daf4c64a9261c4297b3ff9c/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:03.659507 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d8259-97de-4ef1-8fa5-a1fd7093ce1d service placement] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:03.676033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912d8259-97de-4ef1-8fa5-a1fd7093ce1d service placement] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:03.799579 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3873a46-54de-4eef-9b2b-b0392d2064f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:03.800448 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3873a46-54de-4eef-9b2b-b0392d2064f0 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:03.800737 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3873a46-54de-4eef-9b2b-b0392d2064f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:03.801701 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3873a46-54de-4eef-9b2b-b0392d2064f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:03.801899 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3873a46-54de-4eef-9b2b-b0392d2064f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:03.803675 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3873a46-54de-4eef-9b2b-b0392d2064f0 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:03.818215 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 842/1711] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:03 2022] GET /identity/v3/roles => generated 1427 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:03.822813 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59b276-c255-4178-b6b4-2ee10097cabd admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:03.838131 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59b276-c255-4178-b6b4-2ee10097cabd admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:03.852229 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bed93c-aa5e-4ca2-a622-d6b4094024a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:03.856142 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bed93c-aa5e-4ca2-a622-d6b4094024a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:03.856485 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bed93c-aa5e-4ca2-a622-d6b4094024a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:03.857336 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bed93c-aa5e-4ca2-a622-d6b4094024a0 admin admin] PATH_INFO: `/v3/projects/85e2e21340eb4754be5a6bbe2e56905a/users/78a9147f3daf4c64a9261c4297b3ff9c/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:03.864455 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bed93c-aa5e-4ca2-a622-d6b4094024a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85e2e21340eb4754be5a6bbe2e56905a, user_id=78a9147f3daf4c64a9261c4297b3ff9c, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:03.870455 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bed93c-aa5e-4ca2-a622-d6b4094024a0 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:03.908441 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 870/1712] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:03 2022] PUT /identity/v3/projects/85e2e21340eb4754be5a6bbe2e56905a/users/78a9147f3daf4c64a9261c4297b3ff9c/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:03.920010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff2661-e360-4ed9-89a7-6b951f7d666a None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:03.920294 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff2661-e360-4ed9-89a7-6b951f7d666a None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:03.920894 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff2661-e360-4ed9-89a7-6b951f7d666a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:03.979486 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-44ff2661-e360-4ed9-89a7-6b951f7d666a None None] MFA Rules not processed for user `78a9147f3daf4c64a9261c4297b3ff9c`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:03.980788 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-44ff2661-e360-4ed9-89a7-6b951f7d666a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:04.006717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:04.111106 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 843/1713] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:03 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:04.125708 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bed93c-aa5e-4ca2-a622-d6b4094024a0 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:04.281626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bed93c-aa5e-4ca2-a622-d6b4094024a0 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:04.293601 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c81f63-f7e5-4ca2-a05c-fca7ffb27566 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:04.295463 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c81f63-f7e5-4ca2-a05c-fca7ffb27566 service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:04.296850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c81f63-f7e5-4ca2-a05c-fca7ffb27566 service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:04.298730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c81f63-f7e5-4ca2-a05c-fca7ffb27566 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:04.318514 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c81f63-f7e5-4ca2-a05c-fca7ffb27566 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:04.321732 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c81f63-f7e5-4ca2-a05c-fca7ffb27566 service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:04.343111 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 871/1714] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:04 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:06.314665 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff2661-e360-4ed9-89a7-6b951f7d666a None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:06.327961 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-44ff2661-e360-4ed9-89a7-6b951f7d666a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:06.453419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2010961-4005-4ca7-97b8-208d3628adaf None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:06.455552 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2010961-4005-4ca7-97b8-208d3628adaf None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:06.456044 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2010961-4005-4ca7-97b8-208d3628adaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:06.580801 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-e2010961-4005-4ca7-97b8-208d3628adaf None None] MFA Rules not processed for user `822e6ac5ec8347c9815c770738509d30`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:06.582410 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-e2010961-4005-4ca7-97b8-208d3628adaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:06.669310 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ff2661-e360-4ed9-89a7-6b951f7d666a None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:06.690458 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79e908-b79c-4e68-a4da-3cb61ec2ce8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:06.691297 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79e908-b79c-4e68-a4da-3cb61ec2ce8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:06.691584 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79e908-b79c-4e68-a4da-3cb61ec2ce8a admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:06.691862 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef79e908-b79c-4e68-a4da-3cb61ec2ce8a admin admin] PATH_INFO: `/v3/users/b7fc4b12f4f541609ddf148c8b51d8f1` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:06.693551 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef79e908-b79c-4e68-a4da-3cb61ec2ce8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7fc4b12f4f541609ddf148c8b51d8f1)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:06.696023 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef79e908-b79c-4e68-a4da-3cb61ec2ce8a admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:06.918010 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 872/1715] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:06 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:06.922621 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ef79e908-b79c-4e68-a4da-3cb61ec2ce8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7fc4b12f4f541609ddf148c8b51d8f1'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:06.938234 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750797b-b98d-4ebd-be9c-7945d7e1c2d8 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:06.938907 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750797b-b98d-4ebd-be9c-7945d7e1c2d8 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:06.939384 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4750797b-b98d-4ebd-be9c-7945d7e1c2d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:06.954806 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ef79e908-b79c-4e68-a4da-3cb61ec2ce8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7fc4b12f4f541609ddf148c8b51d8f1'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:06.959189 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ef79e908-b79c-4e68-a4da-3cb61ec2ce8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7fc4b12f4f541609ddf148c8b51d8f1'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:07.005055 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-4750797b-b98d-4ebd-be9c-7945d7e1c2d8 None None] MFA Rules not processed for user `822e6ac5ec8347c9815c770738509d30`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:07.008703 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-4750797b-b98d-4ebd-be9c-7945d7e1c2d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:07.020787 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 873/1716] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:06 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:07.021917 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750797b-b98d-4ebd-be9c-7945d7e1c2d8 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:07.036648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ef79e908-b79c-4e68-a4da-3cb61ec2ce8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7fc4b12f4f541609ddf148c8b51d8f1'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:07.074168 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4750797b-b98d-4ebd-be9c-7945d7e1c2d8 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:07.076078 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ef79e908-b79c-4e68-a4da-3cb61ec2ce8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7fc4b12f4f541609ddf148c8b51d8f1'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:07.098213 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980602a-b6b4-4b6e-8765-201b935598ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:07.102500 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980602a-b6b4-4b6e-8765-201b935598ab service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:07.106361 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980602a-b6b4-4b6e-8765-201b935598ab service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:07.108598 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980602a-b6b4-4b6e-8765-201b935598ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:07.109529 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 844/1717] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:55:06 2022] DELETE /identity/v3/users/b7fc4b12f4f541609ddf148c8b51d8f1 => generated 0 bytes in 792 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:07.110566 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79e908-b79c-4e68-a4da-3cb61ec2ce8a admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:07.112073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-ef79e908-b79c-4e68-a4da-3cb61ec2ce8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:07.116217 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-3980602a-b6b4-4b6e-8765-201b935598ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:07.213173 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:07.290799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:07.343407 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980602a-b6b4-4b6e-8765-201b935598ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:07.345585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980602a-b6b4-4b6e-8765-201b935598ab service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:07.373413 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 874/1718] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:07 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:07.383871 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef79e908-b79c-4e68-a4da-3cb61ec2ce8a admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:07.397144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aadd93-a475-4643-9843-9b390e8c5c10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:07.397973 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aadd93-a475-4643-9843-9b390e8c5c10 service nova] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:07.398550 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aadd93-a475-4643-9843-9b390e8c5c10 service nova] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:07.399019 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aadd93-a475-4643-9843-9b390e8c5c10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:07.406729 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-94aadd93-a475-4643-9843-9b390e8c5c10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:07.614470 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aadd93-a475-4643-9843-9b390e8c5c10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:07.616333 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aadd93-a475-4643-9843-9b390e8c5c10 service nova] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:07.655835 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 845/1719] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:07 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:08.682113 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980602a-b6b4-4b6e-8765-201b935598ab service cinder] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:08.683825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-3980602a-b6b4-4b6e-8765-201b935598ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:08.728074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861a4e1-220c-472a-83c7-ff91e9ee95f2 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:08.728539 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980602a-b6b4-4b6e-8765-201b935598ab service cinder] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:08.728641 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861a4e1-220c-472a-83c7-ff91e9ee95f2 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:08.728641 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5861a4e1-220c-472a-83c7-ff91e9ee95f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:08.804407 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-5861a4e1-220c-472a-83c7-ff91e9ee95f2 None None] MFA Rules not processed for user `53bc14ada1564911b2f0a698aef3f333`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:08.805751 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-5861a4e1-220c-472a-83c7-ff91e9ee95f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:08.937404 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b054b-3eba-4ada-ad9b-78a7ec132c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:08.937404 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b054b-3eba-4ada-ad9b-78a7ec132c24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:08.937404 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b054b-3eba-4ada-ad9b-78a7ec132c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:08.937404 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b054b-3eba-4ada-ad9b-78a7ec132c24 admin admin] PATH_INFO: `/v3/projects/e5e49c7e12b64966a1c1b92a50ba8347` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:08.940399 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068b054b-3eba-4ada-ad9b-78a7ec132c24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5e49c7e12b64966a1c1b92a50ba8347)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:08.945526 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068b054b-3eba-4ada-ad9b-78a7ec132c24 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:09.043518 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 846/1720] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:08 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:09.045156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef6431-0f91-4cf1-ba91-f1a71f943118 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:09.045514 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef6431-0f91-4cf1-ba91-f1a71f943118 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:09.048496 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef6431-0f91-4cf1-ba91-f1a71f943118 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:09.051958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 847/1721] 10.222.0.115 () {54 vars in 917 bytes} [Tue Aug 2 10:55:09 2022] GET /identity => generated 271 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:55:09.068043 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b6713-4846-4c9b-9062-20c21b5b38fb None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:09.068336 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b6713-4846-4c9b-9062-20c21b5b38fb None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:09.068633 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b6713-4846-4c9b-9062-20c21b5b38fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:09.132740 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-068b054b-3eba-4ada-ad9b-78a7ec132c24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5e49c7e12b64966a1c1b92a50ba8347', 'request_id': 'req-068b054b-3eba-4ada-ad9b-78a7ec132c24'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:09.163607 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-0f2b6713-4846-4c9b-9062-20c21b5b38fb None None] MFA Rules not processed for user `98dcb74152ec4375a586ea526e7f07c8`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:09.165672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-0f2b6713-4846-4c9b-9062-20c21b5b38fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:09.166661 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-068b054b-3eba-4ada-ad9b-78a7ec132c24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5e49c7e12b64966a1c1b92a50ba8347', 'request_id': 'req-068b054b-3eba-4ada-ad9b-78a7ec132c24'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:09.258281 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:09.265305 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-068b054b-3eba-4ada-ad9b-78a7ec132c24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5e49c7e12b64966a1c1b92a50ba8347', 'request_id': 'req-068b054b-3eba-4ada-ad9b-78a7ec132c24'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:09.288680 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-068b054b-3eba-4ada-ad9b-78a7ec132c24 admin admin] The token cache is being invalidate because project e5e49c7e12b64966a1c1b92a50ba8347 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:55:09.288980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-068b054b-3eba-4ada-ad9b-78a7ec132c24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:09.291168 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-068b054b-3eba-4ada-ad9b-78a7ec132c24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:09.302968 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 875/1722] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:55:08 2022] DELETE /identity/v3/projects/e5e49c7e12b64966a1c1b92a50ba8347 => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:09.304084 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b054b-3eba-4ada-ad9b-78a7ec132c24 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:09.308914 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-068b054b-3eba-4ada-ad9b-78a7ec132c24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:09.325574 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 848/1723] 10.222.0.115 () {58 vars in 1014 bytes} [Tue Aug 2 10:55:09 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:55:09.328720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b6713-4846-4c9b-9062-20c21b5b38fb None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:09.355475 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-0f2b6713-4846-4c9b-9062-20c21b5b38fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:09.578730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b054b-3eba-4ada-ad9b-78a7ec132c24 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:09.587223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2b6713-4846-4c9b-9062-20c21b5b38fb None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:09.594751 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311f617b-4706-4cb6-ba81-18d7ee483a12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:09.595715 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f617b-4706-4cb6-ba81-18d7ee483a12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:09.598104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f617b-4706-4cb6-ba81-18d7ee483a12 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:09.598104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311f617b-4706-4cb6-ba81-18d7ee483a12 admin admin] PATH_INFO: `/v3/users/fbf5248a340944bb831f446f97426409` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:09.598322 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902649c-6b37-442f-b262-f83d6e504733 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:09.598322 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902649c-6b37-442f-b262-f83d6e504733 service nova] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:09.598322 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902649c-6b37-442f-b262-f83d6e504733 service nova] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:09.598592 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f617b-4706-4cb6-ba81-18d7ee483a12 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fbf5248a340944bb831f446f97426409)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:09.598674 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d902649c-6b37-442f-b262-f83d6e504733 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:09.601095 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311f617b-4706-4cb6-ba81-18d7ee483a12 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:09.607680 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-d902649c-6b37-442f-b262-f83d6e504733 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:09.633843 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:09.765257 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902649c-6b37-442f-b262-f83d6e504733 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:09.766000 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d902649c-6b37-442f-b262-f83d6e504733 service nova] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:09.777329 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 876/1724] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:09 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:09.794613 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902649c-6b37-442f-b262-f83d6e504733 service nova] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:09.798904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-d902649c-6b37-442f-b262-f83d6e504733 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:09.836921 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d902649c-6b37-442f-b262-f83d6e504733 service nova] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:09.846711 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8013c1-5393-45f5-ade2-0d5c4cc393cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:09.846711 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8013c1-5393-45f5-ade2-0d5c4cc393cb service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:09.846711 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8013c1-5393-45f5-ade2-0d5c4cc393cb service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:09.847138 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8013c1-5393-45f5-ade2-0d5c4cc393cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:09.862457 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8013c1-5393-45f5-ade2-0d5c4cc393cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:09.882730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8013c1-5393-45f5-ade2-0d5c4cc393cb service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:09.922920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-311f617b-4706-4cb6-ba81-18d7ee483a12 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fbf5248a340944bb831f446f97426409'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:09.931004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 877/1725] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:09 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:09.938769 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-311f617b-4706-4cb6-ba81-18d7ee483a12 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fbf5248a340944bb831f446f97426409'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:09.940712 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-311f617b-4706-4cb6-ba81-18d7ee483a12 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fbf5248a340944bb831f446f97426409'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:10.031505 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-311f617b-4706-4cb6-ba81-18d7ee483a12 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fbf5248a340944bb831f446f97426409'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:10.086706 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-311f617b-4706-4cb6-ba81-18d7ee483a12 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fbf5248a340944bb831f446f97426409'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:10.118894 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 849/1726] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:55:09 2022] DELETE /identity/v3/users/fbf5248a340944bb831f446f97426409 => generated 0 bytes in 793 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:11.294303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85524ddb-84e5-4beb-9d7a-ab2cd11e7138 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:11.295626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85524ddb-84e5-4beb-9d7a-ab2cd11e7138 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:11.296137 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85524ddb-84e5-4beb-9d7a-ab2cd11e7138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:11.366113 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-85524ddb-84e5-4beb-9d7a-ab2cd11e7138 None None] MFA Rules not processed for user `ab1211acdd9a4df48c46f8807ed00063`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:11.370242 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-85524ddb-84e5-4beb-9d7a-ab2cd11e7138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:11.431291 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c0f21-01ed-432e-9cc8-673ac598886f None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:11.432367 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c0f21-01ed-432e-9cc8-673ac598886f None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:11.432802 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947c0f21-01ed-432e-9cc8-673ac598886f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:11.528358 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-947c0f21-01ed-432e-9cc8-673ac598886f None None] MFA Rules not processed for user `f3f41f17f5154e01a699043edc350cf8`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:11.531186 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-947c0f21-01ed-432e-9cc8-673ac598886f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:11.590177 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 878/1727] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:11 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:11.591206 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85524ddb-84e5-4beb-9d7a-ab2cd11e7138 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:11.594264 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-85524ddb-84e5-4beb-9d7a-ab2cd11e7138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:11.694785 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:11.724164 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 850/1728] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:55:11.725010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c0f21-01ed-432e-9cc8-673ac598886f None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:11.736965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-947c0f21-01ed-432e-9cc8-673ac598886f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:11.777628 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85524ddb-84e5-4beb-9d7a-ab2cd11e7138 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:11.790730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8301c-f519-4126-804b-10edc5bd37d8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc4b962e2a742b08beef7187cc527e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:11.791403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8301c-f519-4126-804b-10edc5bd37d8 service placement] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:11.791681 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8301c-f519-4126-804b-10edc5bd37d8 service placement] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:11.792108 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd8301c-f519-4126-804b-10edc5bd37d8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:11.794509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-0dd8301c-f519-4126-804b-10edc5bd37d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:11.820915 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8301c-f519-4126-804b-10edc5bd37d8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:11.822722 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd8301c-f519-4126-804b-10edc5bd37d8 service placement] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:11.846654 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 879/1729] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:11 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:11.847729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8301c-f519-4126-804b-10edc5bd37d8 service placement] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:11.854404 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-0dd8301c-f519-4126-804b-10edc5bd37d8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:11.888186 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947c0f21-01ed-432e-9cc8-673ac598886f None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:11.902168 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed222c-0ff5-4171-8b7e-97d3f2a3f704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:11.902835 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed222c-0ff5-4171-8b7e-97d3f2a3f704 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:11.903228 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed222c-0ff5-4171-8b7e-97d3f2a3f704 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:11.907490 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:11.907784 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ed222c-0ff5-4171-8b7e-97d3f2a3f704 admin admin] PATH_INFO: `/v3/projects/93d71cb6f13b470783204fc17f7a03c4` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:11.916070 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed222c-0ff5-4171-8b7e-97d3f2a3f704 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93d71cb6f13b470783204fc17f7a03c4)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:11.918055 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ed222c-0ff5-4171-8b7e-97d3f2a3f704 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:12.057007 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd8301c-f519-4126-804b-10edc5bd37d8 service placement] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:12.068061 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-e7ed222c-0ff5-4171-8b7e-97d3f2a3f704 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93d71cb6f13b470783204fc17f7a03c4', 'request_id': 'req-e7ed222c-0ff5-4171-8b7e-97d3f2a3f704'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:12.174024 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-e7ed222c-0ff5-4171-8b7e-97d3f2a3f704 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93d71cb6f13b470783204fc17f7a03c4', 'request_id': 'req-e7ed222c-0ff5-4171-8b7e-97d3f2a3f704'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:12.183410 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-e7ed222c-0ff5-4171-8b7e-97d3f2a3f704 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93d71cb6f13b470783204fc17f7a03c4', 'request_id': 'req-e7ed222c-0ff5-4171-8b7e-97d3f2a3f704'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:12.184652 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a15ca94-97b6-4e22-a9cb-8f061517557b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:12.190708 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15ca94-97b6-4e22-a9cb-8f061517557b service nova] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:12.191205 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15ca94-97b6-4e22-a9cb-8f061517557b service nova] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:12.191796 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a15ca94-97b6-4e22-a9cb-8f061517557b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:12.202917 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-e7ed222c-0ff5-4171-8b7e-97d3f2a3f704 admin admin] The token cache is being invalidate because project 93d71cb6f13b470783204fc17f7a03c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:55:12.204715 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-e7ed222c-0ff5-4171-8b7e-97d3f2a3f704 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:12.205315 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-e7ed222c-0ff5-4171-8b7e-97d3f2a3f704 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:12.210522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 851/1730] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:55:11 2022] DELETE /identity/v3/projects/93d71cb6f13b470783204fc17f7a03c4 => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:12.211630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed222c-0ff5-4171-8b7e-97d3f2a3f704 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:12.214727 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-e7ed222c-0ff5-4171-8b7e-97d3f2a3f704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:12.247547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ed222c-0ff5-4171-8b7e-97d3f2a3f704 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:12.261151 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2094a274-dd6f-4a98-800d-9076c6798ee0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:12.263279 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094a274-dd6f-4a98-800d-9076c6798ee0 service nova] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:12.263992 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094a274-dd6f-4a98-800d-9076c6798ee0 service nova] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:12.265987 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2094a274-dd6f-4a98-800d-9076c6798ee0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:12.270529 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-2094a274-dd6f-4a98-800d-9076c6798ee0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:12.318954 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a15ca94-97b6-4e22-a9cb-8f061517557b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:12.321533 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a15ca94-97b6-4e22-a9cb-8f061517557b service nova] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:12.344960 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 880/1731] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:11 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:12.347861 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f2226-98e5-4cde-930f-0e503b294829 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:12.348493 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f2226-98e5-4cde-930f-0e503b294829 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:12.348923 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f2226-98e5-4cde-930f-0e503b294829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:12.378157 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2094a274-dd6f-4a98-800d-9076c6798ee0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:12.380097 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2094a274-dd6f-4a98-800d-9076c6798ee0 service nova] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:12.388359 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-b39f2226-98e5-4cde-930f-0e503b294829 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:12.393994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-b39f2226-98e5-4cde-930f-0e503b294829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:12.421274 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 852/1732] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:12.498211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:12.536019 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c1f9d-f3ba-4633-b685-10ed23f8b2ed None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:12.538297 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 881/1733] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:12 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:12.539358 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c1f9d-f3ba-4633-b685-10ed23f8b2ed None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:12.539899 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87c1f9d-f3ba-4633-b685-10ed23f8b2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:12.563671 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-f87c1f9d-f3ba-4633-b685-10ed23f8b2ed None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:12.564897 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f87c1f9d-f3ba-4633-b685-10ed23f8b2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:12.574600 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 853/1734] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:12 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:12.589815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f2226-98e5-4cde-930f-0e503b294829 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:12.638365 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f2226-98e5-4cde-930f-0e503b294829 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:12.654306 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd1165f-e7e3-4cc7-af7a-0791c59b240c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:12.655764 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1165f-e7e3-4cc7-af7a-0791c59b240c admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:12.656128 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1165f-e7e3-4cc7-af7a-0791c59b240c admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:12.656476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd1165f-e7e3-4cc7-af7a-0791c59b240c admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:12.657328 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd1165f-e7e3-4cc7-af7a-0791c59b240c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:55:12.657703 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd1165f-e7e3-4cc7-af7a-0791c59b240c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:12.659483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd1165f-e7e3-4cc7-af7a-0791c59b240c admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:12.673587 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 882/1735] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:55:12 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:55:12.678754 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c1f9d-f3ba-4633-b685-10ed23f8b2ed None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:12.689603 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87c1f9d-f3ba-4633-b685-10ed23f8b2ed None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:12.699085 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac0bd9-f2bc-403d-9be7-f9655097c360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:12.699978 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac0bd9-f2bc-403d-9be7-f9655097c360 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:12.703960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac0bd9-f2bc-403d-9be7-f9655097c360 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:12.705628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ac0bd9-f2bc-403d-9be7-f9655097c360 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:12.705628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ac0bd9-f2bc-403d-9be7-f9655097c360 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:12.707982 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ac0bd9-f2bc-403d-9be7-f9655097c360 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:12.860120 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:12.872693 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 854/1736] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:55:12 2022] POST /identity/v3/projects => generated 380 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:12.874526 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd1165f-e7e3-4cc7-af7a-0791c59b240c admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:12.901654 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd1165f-e7e3-4cc7-af7a-0791c59b240c admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:13.047158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f815bf3-c634-4a6d-b31e-acb16ee5b34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:13.047158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f815bf3-c634-4a6d-b31e-acb16ee5b34d admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:13.047158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f815bf3-c634-4a6d-b31e-acb16ee5b34d admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:13.047158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f815bf3-c634-4a6d-b31e-acb16ee5b34d admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:13.049678 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f815bf3-c634-4a6d-b31e-acb16ee5b34d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:13.054483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f815bf3-c634-4a6d-b31e-acb16ee5b34d admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:13.232863 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 883/1737] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:55:12 2022] POST /identity/v3/users => generated 291 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:13.234969 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac0bd9-f2bc-403d-9be7-f9655097c360 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:13.247957 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ac0bd9-f2bc-403d-9be7-f9655097c360 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:13.254160 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5002be7-34f2-4c7d-a52f-bed8aaf7817f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:13.261162 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5002be7-34f2-4c7d-a52f-bed8aaf7817f admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:13.261460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5002be7-34f2-4c7d-a52f-bed8aaf7817f admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:13.261743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5002be7-34f2-4c7d-a52f-bed8aaf7817f admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:13.262751 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5002be7-34f2-4c7d-a52f-bed8aaf7817f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:13.265834 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5002be7-34f2-4c7d-a52f-bed8aaf7817f admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:13.283667 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 855/1738] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:13 2022] GET /identity/v3/roles => generated 1427 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:55:13.285035 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f815bf3-c634-4a6d-b31e-acb16ee5b34d admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:13.298162 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f815bf3-c634-4a6d-b31e-acb16ee5b34d admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:13.303670 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b0b173-0c34-42e1-abcf-0e832f6e5326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:13.304298 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b0b173-0c34-42e1-abcf-0e832f6e5326 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:13.304725 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b0b173-0c34-42e1-abcf-0e832f6e5326 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:13.308824 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b0b173-0c34-42e1-abcf-0e832f6e5326 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:13.308824 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b0b173-0c34-42e1-abcf-0e832f6e5326 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:13.308824 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b0b173-0c34-42e1-abcf-0e832f6e5326 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:13.330317 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 884/1739] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:13 2022] GET /identity/v3/roles => generated 1427 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:13.334856 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5002be7-34f2-4c7d-a52f-bed8aaf7817f admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:13.371006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5002be7-34f2-4c7d-a52f-bed8aaf7817f admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:13.379280 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c02ae0-2aed-4de9-bf34-c85be40c0663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:13.380450 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c02ae0-2aed-4de9-bf34-c85be40c0663 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:13.381112 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c02ae0-2aed-4de9-bf34-c85be40c0663 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:13.381580 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c02ae0-2aed-4de9-bf34-c85be40c0663 admin admin] PATH_INFO: `/v3/projects/2a013a5b22bd481b8967bc1d565de2ef/users/995b34126c9f42eeb16aa7bc6ba3d912/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:13.428629 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c02ae0-2aed-4de9-bf34-c85be40c0663 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a013a5b22bd481b8967bc1d565de2ef, user_id=995b34126c9f42eeb16aa7bc6ba3d912, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:13.451251 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c02ae0-2aed-4de9-bf34-c85be40c0663 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:13.593629 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 856/1740] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:13 2022] PUT /identity/v3/projects/2a013a5b22bd481b8967bc1d565de2ef/users/995b34126c9f42eeb16aa7bc6ba3d912/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:13.596196 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5567a-9675-49e3-9eae-3ae4db94cffb None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:13.596596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5567a-9675-49e3-9eae-3ae4db94cffb None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:13.596756 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f5567a-9675-49e3-9eae-3ae4db94cffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:13.753153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-06f5567a-9675-49e3-9eae-3ae4db94cffb None None] MFA Rules not processed for user `995b34126c9f42eeb16aa7bc6ba3d912`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:13.759727 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-06f5567a-9675-49e3-9eae-3ae4db94cffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:14.004895 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 885/1741] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:13 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:14.010692 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c02ae0-2aed-4de9-bf34-c85be40c0663 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:14.209049 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c02ae0-2aed-4de9-bf34-c85be40c0663 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:14.253259 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086883a8-471d-4643-b0a9-43914930aac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:14.256682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086883a8-471d-4643-b0a9-43914930aac4 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:14.256980 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086883a8-471d-4643-b0a9-43914930aac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:14.257259 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086883a8-471d-4643-b0a9-43914930aac4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:14.261754 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086883a8-471d-4643-b0a9-43914930aac4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:14.265301 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086883a8-471d-4643-b0a9-43914930aac4 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:14.429259 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 857/1742] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:55:14 2022] POST /identity/v3/projects => generated 382 bytes in 421 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:55:14.433124 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5567a-9675-49e3-9eae-3ae4db94cffb None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:14.489732 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f5567a-9675-49e3-9eae-3ae4db94cffb None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:14.656053 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:14.717621 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9fb205-5d83-4654-a6f4-885b090bf425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:14.722528 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9fb205-5d83-4654-a6f4-885b090bf425 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:14.722945 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9fb205-5d83-4654-a6f4-885b090bf425 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:14.723302 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9fb205-5d83-4654-a6f4-885b090bf425 admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:14.725313 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9fb205-5d83-4654-a6f4-885b090bf425 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:14.731483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9fb205-5d83-4654-a6f4-885b090bf425 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:14.885460 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 886/1743] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:55:14 2022] POST /identity/v3/users => generated 298 bytes in 453 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:55:14.891320 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086883a8-471d-4643-b0a9-43914930aac4 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:14.906191 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086883a8-471d-4643-b0a9-43914930aac4 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:14.924572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9811f9-149d-4ae7-8ad0-3403b96ecca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:14.926016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9811f9-149d-4ae7-8ad0-3403b96ecca9 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:14.926485 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9811f9-149d-4ae7-8ad0-3403b96ecca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:14.926911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9811f9-149d-4ae7-8ad0-3403b96ecca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:14.927878 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9811f9-149d-4ae7-8ad0-3403b96ecca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:14.929895 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9811f9-149d-4ae7-8ad0-3403b96ecca9 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:14.974258 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 858/1744] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:14 2022] GET /identity/v3/roles => generated 1427 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:14.990732 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9fb205-5d83-4654-a6f4-885b090bf425 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:14.993413 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9fb205-5d83-4654-a6f4-885b090bf425 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:15.011627 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58270933-6c79-4c35-a44a-1839123496f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:15.012329 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58270933-6c79-4c35-a44a-1839123496f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:15.012617 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58270933-6c79-4c35-a44a-1839123496f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:15.012894 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58270933-6c79-4c35-a44a-1839123496f1 admin admin] PATH_INFO: `/v3/projects/c3552c8e478a450da459b5546f5de039/users/f2aa724a18564533aaf7b56d4aaa64c3/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:15.053762 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58270933-6c79-4c35-a44a-1839123496f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3552c8e478a450da459b5546f5de039, user_id=f2aa724a18564533aaf7b56d4aaa64c3, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:15.056970 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58270933-6c79-4c35-a44a-1839123496f1 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:15.154704 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 887/1745] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:14 2022] PUT /identity/v3/projects/c3552c8e478a450da459b5546f5de039/users/f2aa724a18564533aaf7b56d4aaa64c3/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:15.159415 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9811f9-149d-4ae7-8ad0-3403b96ecca9 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:15.192457 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9811f9-149d-4ae7-8ad0-3403b96ecca9 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:15.381714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3188512-7f2f-487e-93c0-cd5eca5624c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:15.383135 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3188512-7f2f-487e-93c0-cd5eca5624c3 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:15.383715 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3188512-7f2f-487e-93c0-cd5eca5624c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:15.384139 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3188512-7f2f-487e-93c0-cd5eca5624c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:15.385080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3188512-7f2f-487e-93c0-cd5eca5624c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:15.387191 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3188512-7f2f-487e-93c0-cd5eca5624c3 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:15.414344 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 859/1746] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:15 2022] GET /identity/v3/roles => generated 1427 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:15.416232 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58270933-6c79-4c35-a44a-1839123496f1 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:15.431286 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58270933-6c79-4c35-a44a-1839123496f1 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:15.444652 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd047d32-6b86-4d38-81b1-2df72f0f25a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:15.445430 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd047d32-6b86-4d38-81b1-2df72f0f25a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:15.445863 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd047d32-6b86-4d38-81b1-2df72f0f25a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:15.446182 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd047d32-6b86-4d38-81b1-2df72f0f25a2 admin admin] PATH_INFO: `/v3/projects/c3552c8e478a450da459b5546f5de039/users/f2aa724a18564533aaf7b56d4aaa64c3/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:15.449004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd047d32-6b86-4d38-81b1-2df72f0f25a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3552c8e478a450da459b5546f5de039, user_id=f2aa724a18564533aaf7b56d4aaa64c3, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:15.460506 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd047d32-6b86-4d38-81b1-2df72f0f25a2 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:15.478527 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 888/1747] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:15 2022] PUT /identity/v3/projects/c3552c8e478a450da459b5546f5de039/users/f2aa724a18564533aaf7b56d4aaa64c3/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:15.489463 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d5d9d-b93d-4789-ac10-2ec9ead03a30 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:15.490346 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d5d9d-b93d-4789-ac10-2ec9ead03a30 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:15.490797 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6d5d9d-b93d-4789-ac10-2ec9ead03a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:15.569552 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccce7c8-54da-41b0-86a8-b85d5f2548a6 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:15.570121 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccce7c8-54da-41b0-86a8-b85d5f2548a6 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:15.570657 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccce7c8-54da-41b0-86a8-b85d5f2548a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:15.599689 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-9f6d5d9d-b93d-4789-ac10-2ec9ead03a30 None None] MFA Rules not processed for user `f2aa724a18564533aaf7b56d4aaa64c3`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:15.602060 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-9f6d5d9d-b93d-4789-ac10-2ec9ead03a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:15.610236 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-bccce7c8-54da-41b0-86a8-b85d5f2548a6 None None] MFA Rules not processed for user `53bc14ada1564911b2f0a698aef3f333`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:15.615521 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-bccce7c8-54da-41b0-86a8-b85d5f2548a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:15.661213 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:15.733739 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 889/1748] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:15 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:15.786810 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 860/1749] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:15 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:55:15.802880 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccce7c8-54da-41b0-86a8-b85d5f2548a6 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:15.805408 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-bccce7c8-54da-41b0-86a8-b85d5f2548a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:15.931206 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccce7c8-54da-41b0-86a8-b85d5f2548a6 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:15.945353 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dd40a5-406f-4273-9b49-8b21c555deb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:15.947647 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd40a5-406f-4273-9b49-8b21c555deb6 service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:15.951651 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd40a5-406f-4273-9b49-8b21c555deb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:15.952528 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dd40a5-406f-4273-9b49-8b21c555deb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:15.979450 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dd40a5-406f-4273-9b49-8b21c555deb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:15.984976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dd40a5-406f-4273-9b49-8b21c555deb6 service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:16.020186 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 890/1750] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:15 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:16.387521 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d5d9d-b93d-4789-ac10-2ec9ead03a30 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:16.438399 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6d5d9d-b93d-4789-ac10-2ec9ead03a30 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:16.459009 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ce46f-f421-48c2-901b-e76c0f0ca71d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:16.459692 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ce46f-f421-48c2-901b-e76c0f0ca71d service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:16.459977 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ce46f-f421-48c2-901b-e76c0f0ca71d service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:16.460255 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ce46f-f421-48c2-901b-e76c0f0ca71d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:16.571630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ce46f-f421-48c2-901b-e76c0f0ca71d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:16.573629 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ce46f-f421-48c2-901b-e76c0f0ca71d service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:16.604323 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 861/1751] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:16 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:17.292695 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b9b5e-fb3e-48d0-9d80-448e26db7b69 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:17.293746 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b9b5e-fb3e-48d0-9d80-448e26db7b69 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:17.294581 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21b9b5e-fb3e-48d0-9d80-448e26db7b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:17.417831 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ce46f-f421-48c2-901b-e76c0f0ca71d service cinder] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:17.422135 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-632ce46f-f421-48c2-901b-e76c0f0ca71d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:17.422349 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-b21b9b5e-fb3e-48d0-9d80-448e26db7b69 None None] MFA Rules not processed for user `2b03263937cb4ca7807a24da44d37b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:17.423604 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-b21b9b5e-fb3e-48d0-9d80-448e26db7b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:17.461314 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ce46f-f421-48c2-901b-e76c0f0ca71d service cinder] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:17.472712 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35868eec-cf02-40c5-9081-0c816361256c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:17.473764 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35868eec-cf02-40c5-9081-0c816361256c service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:17.485114 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35868eec-cf02-40c5-9081-0c816361256c service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:17.486423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:17.488708 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35868eec-cf02-40c5-9081-0c816361256c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:17.497711 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-35868eec-cf02-40c5-9081-0c816361256c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:17.615604 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 891/1752] 10.222.0.115 () {58 vars in 1014 bytes} [Tue Aug 2 10:55:17 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:17.653394 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35868eec-cf02-40c5-9081-0c816361256c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:17.657484 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35868eec-cf02-40c5-9081-0c816361256c service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:17.677790 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 862/1753] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:17 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:18.105889 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b9b5e-fb3e-48d0-9d80-448e26db7b69 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:18.108012 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-b21b9b5e-fb3e-48d0-9d80-448e26db7b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:18.379566 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21b9b5e-fb3e-48d0-9d80-448e26db7b69 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:18.408124 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae8df3b-a988-4f4a-8054-aa66361364d5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0daa49b5a35f4a468bb8d0939e7ec20c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:18.409930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8df3b-a988-4f4a-8054-aa66361364d5 service swift] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:18.415633 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8df3b-a988-4f4a-8054-aa66361364d5 service swift] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:18.415633 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae8df3b-a988-4f4a-8054-aa66361364d5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:18.440946 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae8df3b-a988-4f4a-8054-aa66361364d5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:18.443239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae8df3b-a988-4f4a-8054-aa66361364d5 service swift] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:18.471819 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 892/1754] 10.222.0.115 () {60 vars in 1389 bytes} [Tue Aug 2 10:55:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:55:18.851314 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10646828-910f-4b59-b085-3317266dd577 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:18.851876 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10646828-910f-4b59-b085-3317266dd577 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:18.851984 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10646828-910f-4b59-b085-3317266dd577 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:18.913061 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-10646828-910f-4b59-b085-3317266dd577 None None] MFA Rules not processed for user `f3f41f17f5154e01a699043edc350cf8`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:18.925752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-10646828-910f-4b59-b085-3317266dd577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:19.116496 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:19.116496 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 863/1755] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:19.296954 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98e92f-4fa6-4008-a1cb-3de8f4391f19 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:19.297250 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98e92f-4fa6-4008-a1cb-3de8f4391f19 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:19.297535 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b98e92f-4fa6-4008-a1cb-3de8f4391f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:19.360339 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-8b98e92f-4fa6-4008-a1cb-3de8f4391f19 None None] MFA Rules not processed for user `98dcb74152ec4375a586ea526e7f07c8`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:19.365523 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-8b98e92f-4fa6-4008-a1cb-3de8f4391f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:19.551747 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 893/1756] 10.222.0.107 () {58 vars in 1020 bytes} [Tue Aug 2 10:55:19 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:19.580118 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10646828-910f-4b59-b085-3317266dd577 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:19.589661 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-10646828-910f-4b59-b085-3317266dd577 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:19.640511 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10646828-910f-4b59-b085-3317266dd577 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:19.643741 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1f902-a720-4a98-a17e-0ee2c56d79e3 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:19.644599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1f902-a720-4a98-a17e-0ee2c56d79e3 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:19.645347 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed1f902-a720-4a98-a17e-0ee2c56d79e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:19.671318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729bcd98-b9f3-4f1d-9cb3-28e233bf5edf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:19.673123 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729bcd98-b9f3-4f1d-9cb3-28e233bf5edf service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:19.673470 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729bcd98-b9f3-4f1d-9cb3-28e233bf5edf service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:19.673753 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729bcd98-b9f3-4f1d-9cb3-28e233bf5edf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:19.703896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729bcd98-b9f3-4f1d-9cb3-28e233bf5edf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:19.708275 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729bcd98-b9f3-4f1d-9cb3-28e233bf5edf service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:19.721985 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-4ed1f902-a720-4a98-a17e-0ee2c56d79e3 None None] MFA Rules not processed for user `ab1211acdd9a4df48c46f8807ed00063`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:19.726422 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-4ed1f902-a720-4a98-a17e-0ee2c56d79e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:19.757085 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 864/1757] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:19 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:55:19.800531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:19.867994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 894/1758] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:19 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:21.084483 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729bcd98-b9f3-4f1d-9cb3-28e233bf5edf service neutron] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:21.084483 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-729bcd98-b9f3-4f1d-9cb3-28e233bf5edf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:21.229172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729bcd98-b9f3-4f1d-9cb3-28e233bf5edf service neutron] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:21.241271 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f3d49-47a6-41b2-99fc-428911145d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:21.250259 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f3d49-47a6-41b2-99fc-428911145d91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:21.250705 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f3d49-47a6-41b2-99fc-428911145d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:21.251252 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f3d49-47a6-41b2-99fc-428911145d91 admin admin] PATH_INFO: `/v3/users/36e060a0d20749dd85fa18f081d67192` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:21.255243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe9b19-f974-4ccd-9a89-5341f7c33aba None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:21.256168 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe9b19-f974-4ccd-9a89-5341f7c33aba None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:21.256610 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe9b19-f974-4ccd-9a89-5341f7c33aba None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:21.256912 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f3d49-47a6-41b2-99fc-428911145d91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36e060a0d20749dd85fa18f081d67192)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:21.258798 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 895/1759] 10.222.0.115 () {54 vars in 917 bytes} [Tue Aug 2 10:55:21 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:55:21.261464 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f3d49-47a6-41b2-99fc-428911145d91 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:21.269265 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7efd2b-b9d5-491e-905e-4f89d1306263 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:21.275303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7efd2b-b9d5-491e-905e-4f89d1306263 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:21.275303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7efd2b-b9d5-491e-905e-4f89d1306263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:21.317463 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-0b7efd2b-b9d5-491e-905e-4f89d1306263 None None] MFA Rules not processed for user `98dcb74152ec4375a586ea526e7f07c8`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:21.319193 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-0b7efd2b-b9d5-491e-905e-4f89d1306263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:21.330904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 896/1760] 10.222.0.115 () {58 vars in 1014 bytes} [Tue Aug 2 10:55:21 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:55:21.352616 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7efd2b-b9d5-491e-905e-4f89d1306263 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:21.366915 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7efd2b-b9d5-491e-905e-4f89d1306263 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:21.395516 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eff9da-a96d-41a1-b58f-3ed948899872 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:21.398403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eff9da-a96d-41a1-b58f-3ed948899872 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:21.398914 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eff9da-a96d-41a1-b58f-3ed948899872 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:21.399414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eff9da-a96d-41a1-b58f-3ed948899872 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:21.425033 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eff9da-a96d-41a1-b58f-3ed948899872 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:21.426674 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eff9da-a96d-41a1-b58f-3ed948899872 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:21.470660 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 897/1761] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:21 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:21.569266 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-142f3d49-47a6-41b2-99fc-428911145d91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36e060a0d20749dd85fa18f081d67192'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:21.591473 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-142f3d49-47a6-41b2-99fc-428911145d91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36e060a0d20749dd85fa18f081d67192'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:21.597994 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-142f3d49-47a6-41b2-99fc-428911145d91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36e060a0d20749dd85fa18f081d67192'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:21.681235 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-142f3d49-47a6-41b2-99fc-428911145d91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36e060a0d20749dd85fa18f081d67192'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:21.710652 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-142f3d49-47a6-41b2-99fc-428911145d91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36e060a0d20749dd85fa18f081d67192'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:21.736152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 865/1762] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:55:21 2022] DELETE /identity/v3/users/36e060a0d20749dd85fa18f081d67192 => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:21.740896 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487d6b7-a466-48a9-ad88-2817880fef4c None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:21.741643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487d6b7-a466-48a9-ad88-2817880fef4c None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:21.742214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c487d6b7-a466-48a9-ad88-2817880fef4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:21.838835 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-c487d6b7-a466-48a9-ad88-2817880fef4c None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:21.845496 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-c487d6b7-a466-48a9-ad88-2817880fef4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:21.862060 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f3d49-47a6-41b2-99fc-428911145d91 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:21.863775 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-142f3d49-47a6-41b2-99fc-428911145d91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:22.019488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 898/1763] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:21 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:22.081294 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c487d6b7-a466-48a9-ad88-2817880fef4c None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:22.085454 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-c487d6b7-a466-48a9-ad88-2817880fef4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:22.212911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f3d49-47a6-41b2-99fc-428911145d91 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:22.244757 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d6ad1-581c-487d-b994-9b5d5fa98f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:22.245495 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d6ad1-581c-487d-b994-9b5d5fa98f7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:22.245787 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d6ad1-581c-487d-b994-9b5d5fa98f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:22.246059 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d6ad1-581c-487d-b994-9b5d5fa98f7f admin admin] PATH_INFO: `/v3/users/31bc36c4e2874704bb06a571b6073873` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:22.250069 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d6ad1-581c-487d-b994-9b5d5fa98f7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=31bc36c4e2874704bb06a571b6073873)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:22.253013 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d6ad1-581c-487d-b994-9b5d5fa98f7f admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:22.277941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:22.310102 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c487d6b7-a466-48a9-ad88-2817880fef4c None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:22.316989 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7a7097-80e4-4207-9f7c-3ca64fbaf9ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:22.317880 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a7097-80e4-4207-9f7c-3ca64fbaf9ac service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:22.318396 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a7097-80e4-4207-9f7c-3ca64fbaf9ac service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:22.318912 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7a7097-80e4-4207-9f7c-3ca64fbaf9ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:22.332589 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a7097-80e4-4207-9f7c-3ca64fbaf9ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:22.333993 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7a7097-80e4-4207-9f7c-3ca64fbaf9ac service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:22.353987 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 899/1764] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:22 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:22.517717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-5c5d6ad1-581c-487d-b994-9b5d5fa98f7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31bc36c4e2874704bb06a571b6073873'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:22.544264 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-5c5d6ad1-581c-487d-b994-9b5d5fa98f7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31bc36c4e2874704bb06a571b6073873'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:22.551843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-5c5d6ad1-581c-487d-b994-9b5d5fa98f7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31bc36c4e2874704bb06a571b6073873'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:22.627553 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-5c5d6ad1-581c-487d-b994-9b5d5fa98f7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31bc36c4e2874704bb06a571b6073873'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:22.651727 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-5c5d6ad1-581c-487d-b994-9b5d5fa98f7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31bc36c4e2874704bb06a571b6073873'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:22.699381 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 866/1765] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:55:21 2022] DELETE /identity/v3/users/31bc36c4e2874704bb06a571b6073873 => generated 0 bytes in 838 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:22.704566 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493dd90a-d5f5-4d46-b90e-e44f13d5b447 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:22.704849 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493dd90a-d5f5-4d46-b90e-e44f13d5b447 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:22.705324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493dd90a-d5f5-4d46-b90e-e44f13d5b447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:22.797347 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-493dd90a-d5f5-4d46-b90e-e44f13d5b447 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:22.803324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-493dd90a-d5f5-4d46-b90e-e44f13d5b447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:22.965209 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 900/1766] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:22 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:23.073526 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d6ad1-581c-487d-b994-9b5d5fa98f7f admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:23.075676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-5c5d6ad1-581c-487d-b994-9b5d5fa98f7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:23.220223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d6ad1-581c-487d-b994-9b5d5fa98f7f admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:23.228280 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8c682d-49cd-4604-a46e-ba31016810b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:23.248417 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c682d-49cd-4604-a46e-ba31016810b4 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:23.248984 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c682d-49cd-4604-a46e-ba31016810b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:23.249411 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8c682d-49cd-4604-a46e-ba31016810b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:23.265395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8c682d-49cd-4604-a46e-ba31016810b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:23.267063 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8c682d-49cd-4604-a46e-ba31016810b4 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:23.282944 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 867/1767] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:23 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:23.291098 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c183a2-e5bc-4e07-811c-5d9dcc9e8093 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:23.291098 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c183a2-e5bc-4e07-811c-5d9dcc9e8093 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:23.291098 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c183a2-e5bc-4e07-811c-5d9dcc9e8093 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:23.293500 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 901/1768] 10.222.0.115 () {54 vars in 926 bytes} [Tue Aug 2 10:55:23 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:55:23.356656 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621138c6-7f4b-46ac-b4a8-1ae3fdc1d0e3 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:23.357739 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621138c6-7f4b-46ac-b4a8-1ae3fdc1d0e3 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:23.358221 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621138c6-7f4b-46ac-b4a8-1ae3fdc1d0e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:23.398705 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-621138c6-7f4b-46ac-b4a8-1ae3fdc1d0e3 None None] MFA Rules not processed for user `d0478bd540f446ec810e2091c46ac49c`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:23.402687 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-621138c6-7f4b-46ac-b4a8-1ae3fdc1d0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:23.589460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 868/1769] 10.222.0.115 () {58 vars in 1023 bytes} [Tue Aug 2 10:55:23 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:23.612461 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c183a2-e5bc-4e07-811c-5d9dcc9e8093 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:23.615079 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-35c183a2-e5bc-4e07-811c-5d9dcc9e8093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:23.649394 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:23.650878 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c183a2-e5bc-4e07-811c-5d9dcc9e8093 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:23.665884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6ed5c-c22c-4637-ade2-95fca90deabd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:23.666534 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6ed5c-c22c-4637-ade2-95fca90deabd service nova] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:23.666824 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6ed5c-c22c-4637-ade2-95fca90deabd service nova] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:23.667422 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e6ed5c-c22c-4637-ade2-95fca90deabd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:23.683948 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e6ed5c-c22c-4637-ade2-95fca90deabd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:23.688004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e6ed5c-c22c-4637-ade2-95fca90deabd service nova] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:23.723476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 902/1770] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:23 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:24.425951 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621138c6-7f4b-46ac-b4a8-1ae3fdc1d0e3 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:24.427449 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-621138c6-7f4b-46ac-b4a8-1ae3fdc1d0e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:24.481279 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621138c6-7f4b-46ac-b4a8-1ae3fdc1d0e3 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:24.494193 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578405fe-1de4-46aa-a76a-304247694fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:24.494902 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578405fe-1de4-46aa-a76a-304247694fad admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:24.495380 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578405fe-1de4-46aa-a76a-304247694fad admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:24.495822 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578405fe-1de4-46aa-a76a-304247694fad admin admin] PATH_INFO: `/v3/projects/fb73b626b16c4764b9f7336c0840aae0` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:24.500063 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578405fe-1de4-46aa-a76a-304247694fad admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb73b626b16c4764b9f7336c0840aae0)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:24.505263 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578405fe-1de4-46aa-a76a-304247694fad admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:24.566477 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:24.584937 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6ed5c-c22c-4637-ade2-95fca90deabd service nova] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:24.586945 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-b4e6ed5c-c22c-4637-ade2-95fca90deabd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:24.648254 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e6ed5c-c22c-4637-ade2-95fca90deabd service nova] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:24.667095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16da4705-2a26-4b82-9ead-483e4417d5cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:24.667095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da4705-2a26-4b82-9ead-483e4417d5cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:24.667095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da4705-2a26-4b82-9ead-483e4417d5cc admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:24.667095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da4705-2a26-4b82-9ead-483e4417d5cc admin admin] PATH_INFO: `/v3/users/02795c3a9f9e451281b955bade62d9ac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:24.672055 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16da4705-2a26-4b82-9ead-483e4417d5cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=02795c3a9f9e451281b955bade62d9ac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:24.676000 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: WARNING py.warnings [None req-16da4705-2a26-4b82-9ead-483e4417d5cc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 02 10:55:24.676000 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: warnings.warn(msg) Aug 02 10:55:24.676000 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: Aug 02 10:55:24.677564 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16da4705-2a26-4b82-9ead-483e4417d5cc admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:24.726158 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-578405fe-1de4-46aa-a76a-304247694fad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb73b626b16c4764b9f7336c0840aae0', 'request_id': 'req-578405fe-1de4-46aa-a76a-304247694fad'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:24.836127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-578405fe-1de4-46aa-a76a-304247694fad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb73b626b16c4764b9f7336c0840aae0', 'request_id': 'req-578405fe-1de4-46aa-a76a-304247694fad'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:24.850477 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-578405fe-1de4-46aa-a76a-304247694fad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb73b626b16c4764b9f7336c0840aae0', 'request_id': 'req-578405fe-1de4-46aa-a76a-304247694fad'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:24.864640 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-578405fe-1de4-46aa-a76a-304247694fad admin admin] The token cache is being invalidate because project fb73b626b16c4764b9f7336c0840aae0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:55:24.865190 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-578405fe-1de4-46aa-a76a-304247694fad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:24.865876 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-578405fe-1de4-46aa-a76a-304247694fad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:24.867488 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 869/1771] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:55:24 2022] DELETE /identity/v3/projects/fb73b626b16c4764b9f7336c0840aae0 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:24.876282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578405fe-1de4-46aa-a76a-304247694fad admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:24.880743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-578405fe-1de4-46aa-a76a-304247694fad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:24.954703 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578405fe-1de4-46aa-a76a-304247694fad admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:24.959717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25877b7b-8787-4240-ad9f-d02da37a24bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:24.960634 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25877b7b-8787-4240-ad9f-d02da37a24bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:24.960982 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25877b7b-8787-4240-ad9f-d02da37a24bb admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:24.962476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25877b7b-8787-4240-ad9f-d02da37a24bb admin admin] PATH_INFO: `/v3/projects/bc5aa3b9a1e24cf4bf525f850525f624` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:24.964064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25877b7b-8787-4240-ad9f-d02da37a24bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc5aa3b9a1e24cf4bf525f850525f624)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:24.966543 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25877b7b-8787-4240-ad9f-d02da37a24bb admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:25.017453 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-16da4705-2a26-4b82-9ead-483e4417d5cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02795c3a9f9e451281b955bade62d9ac'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:25.022257 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-16da4705-2a26-4b82-9ead-483e4417d5cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02795c3a9f9e451281b955bade62d9ac'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:25.089436 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-25877b7b-8787-4240-ad9f-d02da37a24bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc5aa3b9a1e24cf4bf525f850525f624', 'request_id': 'req-25877b7b-8787-4240-ad9f-d02da37a24bb'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:25.098966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-16da4705-2a26-4b82-9ead-483e4417d5cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02795c3a9f9e451281b955bade62d9ac'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:25.111844 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-16da4705-2a26-4b82-9ead-483e4417d5cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02795c3a9f9e451281b955bade62d9ac'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:25.140423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-16da4705-2a26-4b82-9ead-483e4417d5cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02795c3a9f9e451281b955bade62d9ac'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:25.147696 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 903/1772] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:55:24 2022] DELETE /identity/v3/users/02795c3a9f9e451281b955bade62d9ac => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:25.150093 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16da4705-2a26-4b82-9ead-483e4417d5cc admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:25.151800 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-16da4705-2a26-4b82-9ead-483e4417d5cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:25.161430 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-25877b7b-8787-4240-ad9f-d02da37a24bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc5aa3b9a1e24cf4bf525f850525f624', 'request_id': 'req-25877b7b-8787-4240-ad9f-d02da37a24bb'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:25.168734 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-25877b7b-8787-4240-ad9f-d02da37a24bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc5aa3b9a1e24cf4bf525f850525f624', 'request_id': 'req-25877b7b-8787-4240-ad9f-d02da37a24bb'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:25.183362 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-25877b7b-8787-4240-ad9f-d02da37a24bb admin admin] The token cache is being invalidate because project bc5aa3b9a1e24cf4bf525f850525f624 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:55:25.183796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-25877b7b-8787-4240-ad9f-d02da37a24bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:25.185071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-25877b7b-8787-4240-ad9f-d02da37a24bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:25.187210 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 870/1773] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:55:24 2022] DELETE /identity/v3/projects/bc5aa3b9a1e24cf4bf525f850525f624 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:25.188438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a2f18-cde7-4454-8755-64d609bb0b5c None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:25.188984 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a2f18-cde7-4454-8755-64d609bb0b5c None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:25.189360 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495a2f18-cde7-4454-8755-64d609bb0b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:25.210902 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-495a2f18-cde7-4454-8755-64d609bb0b5c None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:25.212557 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-495a2f18-cde7-4454-8755-64d609bb0b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:25.238903 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16da4705-2a26-4b82-9ead-483e4417d5cc admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:25.244106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7f071-0e37-4a82-9b7d-388aa6f09731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:25.244815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7f071-0e37-4a82-9b7d-388aa6f09731 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:25.245106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7f071-0e37-4a82-9b7d-388aa6f09731 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:25.245380 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7f071-0e37-4a82-9b7d-388aa6f09731 admin admin] PATH_INFO: `/v3/users/042a99b0f6ae45c1b61c06256284f32b` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:25.248733 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b7f071-0e37-4a82-9b7d-388aa6f09731 admin admin] RBAC: Authorizing `identity:delete_user(user_id=042a99b0f6ae45c1b61c06256284f32b)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:25.250542 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:25.251300 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b7f071-0e37-4a82-9b7d-388aa6f09731 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:25.289219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 871/1774] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:25.290630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f05894-e31b-4098-b2b4-f66753eed6db None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:25.290919 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f05894-e31b-4098-b2b4-f66753eed6db None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:25.291239 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f05894-e31b-4098-b2b4-f66753eed6db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:25.316712 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-94f05894-e31b-4098-b2b4-f66753eed6db None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:25.316712 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-94f05894-e31b-4098-b2b4-f66753eed6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:25.322689 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 872/1775] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:25.323155 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f05894-e31b-4098-b2b4-f66753eed6db None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:25.324437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-94f05894-e31b-4098-b2b4-f66753eed6db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:25.389740 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f05894-e31b-4098-b2b4-f66753eed6db None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:25.394184 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78752500-f058-4105-9285-3e271272dac1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:25.394921 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78752500-f058-4105-9285-3e271272dac1 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:25.395239 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78752500-f058-4105-9285-3e271272dac1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:25.395516 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78752500-f058-4105-9285-3e271272dac1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:25.409965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78752500-f058-4105-9285-3e271272dac1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:25.411078 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78752500-f058-4105-9285-3e271272dac1 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:25.423564 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 873/1776] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:25 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:25.424863 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384b3f8-0e5f-4be2-8b56-09129fb97be6 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:25.425151 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384b3f8-0e5f-4be2-8b56-09129fb97be6 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:25.425430 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0384b3f8-0e5f-4be2-8b56-09129fb97be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:25.455296 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e4b7f071-0e37-4a82-9b7d-388aa6f09731 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '042a99b0f6ae45c1b61c06256284f32b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:25.455296 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e4b7f071-0e37-4a82-9b7d-388aa6f09731 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '042a99b0f6ae45c1b61c06256284f32b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:25.470152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-0384b3f8-0e5f-4be2-8b56-09129fb97be6 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:25.471417 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-0384b3f8-0e5f-4be2-8b56-09129fb97be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:25.484316 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 874/1777] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:25.485320 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384b3f8-0e5f-4be2-8b56-09129fb97be6 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:25.496322 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0384b3f8-0e5f-4be2-8b56-09129fb97be6 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:25.502496 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af8fe1-006d-4be2-bca9-9d709f549bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:25.503466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af8fe1-006d-4be2-bca9-9d709f549bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:25.503889 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af8fe1-006d-4be2-bca9-9d709f549bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:25.504331 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af8fe1-006d-4be2-bca9-9d709f549bd0 admin admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:25.505189 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af8fe1-006d-4be2-bca9-9d709f549bd0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:55:25.505648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af8fe1-006d-4be2-bca9-9d709f549bd0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:25.508382 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af8fe1-006d-4be2-bca9-9d709f549bd0 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:25.523738 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 875/1778] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:55:25 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:55:25.528318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af8fe1-006d-4be2-bca9-9d709f549bd0 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:25.540549 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af8fe1-006d-4be2-bca9-9d709f549bd0 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:25.548269 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb67a0-6fa2-4c71-8aa1-0395b8aa038d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:25.548924 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb67a0-6fa2-4c71-8aa1-0395b8aa038d admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:25.549178 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb67a0-6fa2-4c71-8aa1-0395b8aa038d admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:25.549670 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb67a0-6fa2-4c71-8aa1-0395b8aa038d admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:25.550426 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cb67a0-6fa2-4c71-8aa1-0395b8aa038d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:25.552561 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cb67a0-6fa2-4c71-8aa1-0395b8aa038d admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:25.562487 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e4b7f071-0e37-4a82-9b7d-388aa6f09731 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '042a99b0f6ae45c1b61c06256284f32b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:25.573220 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e4b7f071-0e37-4a82-9b7d-388aa6f09731 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '042a99b0f6ae45c1b61c06256284f32b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:25.577450 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:25.603403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e4b7f071-0e37-4a82-9b7d-388aa6f09731 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '042a99b0f6ae45c1b61c06256284f32b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:25.612774 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 904/1779] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:55:25 2022] DELETE /identity/v3/users/042a99b0f6ae45c1b61c06256284f32b => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:25.734752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 876/1780] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:55:25 2022] POST /identity/v3/projects => generated 406 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:25.739186 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7f071-0e37-4a82-9b7d-388aa6f09731 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:25.743364 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-e4b7f071-0e37-4a82-9b7d-388aa6f09731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:25.921652 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7f071-0e37-4a82-9b7d-388aa6f09731 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:26.040405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d56f2ce-d944-43c3-8296-c599b68f6750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:26.041527 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56f2ce-d944-43c3-8296-c599b68f6750 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:26.041923 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56f2ce-d944-43c3-8296-c599b68f6750 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:26.042268 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56f2ce-d944-43c3-8296-c599b68f6750 admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:26.043140 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d56f2ce-d944-43c3-8296-c599b68f6750 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:26.047186 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d56f2ce-d944-43c3-8296-c599b68f6750 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:26.253465 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 905/1781] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:55:25 2022] POST /identity/v3/users => generated 304 bytes in 511 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:26.253821 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb67a0-6fa2-4c71-8aa1-0395b8aa038d admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:26.253937 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d56f2ce-d944-43c3-8296-c599b68f6750 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:26.255959 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-e6cb67a0-6fa2-4c71-8aa1-0395b8aa038d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:26.265479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d56f2ce-d944-43c3-8296-c599b68f6750 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:26.271004 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb67a0-6fa2-4c71-8aa1-0395b8aa038d admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:26.272863 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fd253-d952-4123-9c04-63e6f4aa8d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:26.273643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fd253-d952-4123-9c04-63e6f4aa8d7a admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:26.273993 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fd253-d952-4123-9c04-63e6f4aa8d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:26.274518 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976fd253-d952-4123-9c04-63e6f4aa8d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:26.279401 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976fd253-d952-4123-9c04-63e6f4aa8d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:26.283773 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976fd253-d952-4123-9c04-63e6f4aa8d7a admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:26.288400 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879255d8-73d9-4078-8c98-4ba5a2ffdd0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:26.288400 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879255d8-73d9-4078-8c98-4ba5a2ffdd0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:26.288400 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879255d8-73d9-4078-8c98-4ba5a2ffdd0b admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:26.288400 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879255d8-73d9-4078-8c98-4ba5a2ffdd0b admin admin] PATH_INFO: `/v3/projects/d709dc6793fb484ea9e293e491c75727` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:26.290214 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879255d8-73d9-4078-8c98-4ba5a2ffdd0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d709dc6793fb484ea9e293e491c75727)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:26.292543 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879255d8-73d9-4078-8c98-4ba5a2ffdd0b admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:26.306647 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 906/1782] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:26 2022] GET /identity/v3/roles => generated 1427 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:26.307711 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fd253-d952-4123-9c04-63e6f4aa8d7a admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:26.311127 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-976fd253-d952-4123-9c04-63e6f4aa8d7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:26.335848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976fd253-d952-4123-9c04-63e6f4aa8d7a admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:26.341300 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730c6ca-3b4a-4a5c-8885-d4496d22cac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:26.342694 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730c6ca-3b4a-4a5c-8885-d4496d22cac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:26.342694 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730c6ca-3b4a-4a5c-8885-d4496d22cac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:26.342694 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0730c6ca-3b4a-4a5c-8885-d4496d22cac4 admin admin] PATH_INFO: `/v3/projects/7a05dc80ed4a4c4b87c1e5c96340a488` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:26.344111 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0730c6ca-3b4a-4a5c-8885-d4496d22cac4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a05dc80ed4a4c4b87c1e5c96340a488)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:26.346449 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0730c6ca-3b4a-4a5c-8885-d4496d22cac4 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:26.477954 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-879255d8-73d9-4078-8c98-4ba5a2ffdd0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd709dc6793fb484ea9e293e491c75727', 'request_id': 'req-879255d8-73d9-4078-8c98-4ba5a2ffdd0b'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:26.511549 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-0730c6ca-3b4a-4a5c-8885-d4496d22cac4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a05dc80ed4a4c4b87c1e5c96340a488', 'request_id': 'req-0730c6ca-3b4a-4a5c-8885-d4496d22cac4'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:26.523511 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-0730c6ca-3b4a-4a5c-8885-d4496d22cac4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a05dc80ed4a4c4b87c1e5c96340a488', 'request_id': 'req-0730c6ca-3b4a-4a5c-8885-d4496d22cac4'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:26.585776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-879255d8-73d9-4078-8c98-4ba5a2ffdd0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd709dc6793fb484ea9e293e491c75727', 'request_id': 'req-879255d8-73d9-4078-8c98-4ba5a2ffdd0b'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:26.601366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-879255d8-73d9-4078-8c98-4ba5a2ffdd0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd709dc6793fb484ea9e293e491c75727', 'request_id': 'req-879255d8-73d9-4078-8c98-4ba5a2ffdd0b'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:26.624409 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-879255d8-73d9-4078-8c98-4ba5a2ffdd0b admin admin] The token cache is being invalidate because project d709dc6793fb484ea9e293e491c75727 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:55:26.624613 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-879255d8-73d9-4078-8c98-4ba5a2ffdd0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:26.625307 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-879255d8-73d9-4078-8c98-4ba5a2ffdd0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:26.626918 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 877/1783] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:55:26 2022] DELETE /identity/v3/projects/d709dc6793fb484ea9e293e491c75727 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:26.628698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879255d8-73d9-4078-8c98-4ba5a2ffdd0b admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:26.630191 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-879255d8-73d9-4078-8c98-4ba5a2ffdd0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:26.647678 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:26.659294 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-0730c6ca-3b4a-4a5c-8885-d4496d22cac4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a05dc80ed4a4c4b87c1e5c96340a488', 'request_id': 'req-0730c6ca-3b4a-4a5c-8885-d4496d22cac4'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:26.662231 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=88,keystone:DELETE=8,keystone:INSERT=4 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:26.687687 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-0730c6ca-3b4a-4a5c-8885-d4496d22cac4 admin admin] The token cache is being invalidate because project 7a05dc80ed4a4c4b87c1e5c96340a488 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:55:26.691191 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-0730c6ca-3b4a-4a5c-8885-d4496d22cac4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:26.691191 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-0730c6ca-3b4a-4a5c-8885-d4496d22cac4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:26.691191 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 907/1784] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:55:26 2022] DELETE /identity/v3/projects/7a05dc80ed4a4c4b87c1e5c96340a488 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:26.691845 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730c6ca-3b4a-4a5c-8885-d4496d22cac4 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:26.700164 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-0730c6ca-3b4a-4a5c-8885-d4496d22cac4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:26.787458 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879255d8-73d9-4078-8c98-4ba5a2ffdd0b admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:26.896619 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0730c6ca-3b4a-4a5c-8885-d4496d22cac4 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:26.920388 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a5e8aa-fbc5-4ac7-aa60-02232dae28f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:26.920388 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5e8aa-fbc5-4ac7-aa60-02232dae28f1 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:26.920388 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5e8aa-fbc5-4ac7-aa60-02232dae28f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:26.920388 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a5e8aa-fbc5-4ac7-aa60-02232dae28f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:26.921047 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a5e8aa-fbc5-4ac7-aa60-02232dae28f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:26.930540 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a5e8aa-fbc5-4ac7-aa60-02232dae28f1 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:26.933208 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95d3370-96f5-4efc-80be-8f5229a57cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:26.933855 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d3370-96f5-4efc-80be-8f5229a57cae admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:26.934144 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d3370-96f5-4efc-80be-8f5229a57cae admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:26.934426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95d3370-96f5-4efc-80be-8f5229a57cae admin admin] PATH_INFO: `/v3/users/bdf2d127118f4d78979c4935da3d1756` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:26.937319 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95d3370-96f5-4efc-80be-8f5229a57cae admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdf2d127118f4d78979c4935da3d1756)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:26.940377 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95d3370-96f5-4efc-80be-8f5229a57cae admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:26.962940 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 878/1785] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:26 2022] GET /identity/v3/roles => generated 1427 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:26.965509 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e73aa-35e1-4375-9417-01c9c48c37c8 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:26.965776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e73aa-35e1-4375-9417-01c9c48c37c8 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:26.966043 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e73aa-35e1-4375-9417-01c9c48c37c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:27.030081 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-f09e73aa-35e1-4375-9417-01c9c48c37c8 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:27.031424 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f09e73aa-35e1-4375-9417-01c9c48c37c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:27.052345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 879/1786] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:26 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:27.052345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e73aa-35e1-4375-9417-01c9c48c37c8 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:27.054530 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f09e73aa-35e1-4375-9417-01c9c48c37c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:27.091660 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09e73aa-35e1-4375-9417-01c9c48c37c8 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:27.109206 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11463296-8f66-4eef-9d04-1a50daaaf6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:27.109966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11463296-8f66-4eef-9d04-1a50daaaf6e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:27.110249 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11463296-8f66-4eef-9d04-1a50daaaf6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:27.110524 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11463296-8f66-4eef-9d04-1a50daaaf6e3 admin admin] PATH_INFO: `/v3/projects/d3f3a751b6dc4a6aa51a204a1eadc7f0/users/f864ba3794464d0cbcd200f6d140aa5c/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:27.138744 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11463296-8f66-4eef-9d04-1a50daaaf6e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3f3a751b6dc4a6aa51a204a1eadc7f0, user_id=f864ba3794464d0cbcd200f6d140aa5c, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:27.142146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11463296-8f66-4eef-9d04-1a50daaaf6e3 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:27.253751 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 880/1787] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:27 2022] PUT /identity/v3/projects/d3f3a751b6dc4a6aa51a204a1eadc7f0/users/f864ba3794464d0cbcd200f6d140aa5c/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:27.256055 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a956f-da28-4dc0-a7a3-e199c6e8f937 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:27.257172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a956f-da28-4dc0-a7a3-e199c6e8f937 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:27.257661 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795a956f-da28-4dc0-a7a3-e199c6e8f937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:27.270069 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-f95d3370-96f5-4efc-80be-8f5229a57cae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdf2d127118f4d78979c4935da3d1756'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:27.271416 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-f95d3370-96f5-4efc-80be-8f5229a57cae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdf2d127118f4d78979c4935da3d1756'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:27.293034 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-795a956f-da28-4dc0-a7a3-e199c6e8f937 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:27.295450 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-795a956f-da28-4dc0-a7a3-e199c6e8f937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:27.392085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-f95d3370-96f5-4efc-80be-8f5229a57cae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdf2d127118f4d78979c4935da3d1756'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:27.405727 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-f95d3370-96f5-4efc-80be-8f5229a57cae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdf2d127118f4d78979c4935da3d1756'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:27.448728 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-f95d3370-96f5-4efc-80be-8f5229a57cae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdf2d127118f4d78979c4935da3d1756'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:27.467447 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 908/1788] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:55:26 2022] DELETE /identity/v3/users/bdf2d127118f4d78979c4935da3d1756 => generated 0 bytes in 773 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:27.467447 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63545861-e201-4289-b880-b2e16e9b594e None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:27.467791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63545861-e201-4289-b880-b2e16e9b594e None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:27.467791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63545861-e201-4289-b880-b2e16e9b594e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:27.525504 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-63545861-e201-4289-b880-b2e16e9b594e None None] MFA Rules not processed for user `f864ba3794464d0cbcd200f6d140aa5c`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:27.525504 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-63545861-e201-4289-b880-b2e16e9b594e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:27.537080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 881/1789] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:27 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:27.541600 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a956f-da28-4dc0-a7a3-e199c6e8f937 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:27.568132 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795a956f-da28-4dc0-a7a3-e199c6e8f937 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:27.671236 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:27.678219 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:27.698927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 909/1790] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:27 2022] POST /identity/v3/auth/tokens => generated 3292 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:27.700149 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63545861-e201-4289-b880-b2e16e9b594e None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:27.704976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-63545861-e201-4289-b880-b2e16e9b594e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:27.786137 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a97e1b-be6a-4f66-a755-49c09910ac3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:27.786818 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a97e1b-be6a-4f66-a755-49c09910ac3d admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:27.787100 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a97e1b-be6a-4f66-a755-49c09910ac3d admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:27.787373 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a97e1b-be6a-4f66-a755-49c09910ac3d admin admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:27.789879 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a97e1b-be6a-4f66-a755-49c09910ac3d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:55:27.790211 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a97e1b-be6a-4f66-a755-49c09910ac3d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:27.791934 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a97e1b-be6a-4f66-a755-49c09910ac3d admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:27.819228 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 882/1791] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:55:27 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:55:27.820248 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a97e1b-be6a-4f66-a755-49c09910ac3d admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:27.834890 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a97e1b-be6a-4f66-a755-49c09910ac3d admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:27.851694 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77945d64-9769-4173-b36b-976eb1261b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:27.852440 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77945d64-9769-4173-b36b-976eb1261b0a admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:27.852798 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77945d64-9769-4173-b36b-976eb1261b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:27.853162 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77945d64-9769-4173-b36b-976eb1261b0a admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:27.853906 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77945d64-9769-4173-b36b-976eb1261b0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:27.856232 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77945d64-9769-4173-b36b-976eb1261b0a admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:27.877140 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63545861-e201-4289-b880-b2e16e9b594e None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:27.886231 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a6988-6772-4c1a-8d9b-ff5d115ec26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:27.887359 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a6988-6772-4c1a-8d9b-ff5d115ec26e admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:27.887704 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a6988-6772-4c1a-8d9b-ff5d115ec26e admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:27.889429 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384a6988-6772-4c1a-8d9b-ff5d115ec26e admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:27.890479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a6988-6772-4c1a-8d9b-ff5d115ec26e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:27.893163 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384a6988-6772-4c1a-8d9b-ff5d115ec26e admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:27.988348 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 910/1792] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:55:27 2022] POST /identity/v3/projects => generated 404 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:27.990517 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a6988-6772-4c1a-8d9b-ff5d115ec26e admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:27.992670 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 883/1793] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:55:27 2022] POST /identity/v3/projects => generated 368 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:27.993621 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77945d64-9769-4173-b36b-976eb1261b0a admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:28.110784 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77945d64-9769-4173-b36b-976eb1261b0a admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:28.168304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384a6988-6772-4c1a-8d9b-ff5d115ec26e admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:28.174563 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd316f-4377-4bcc-801a-c77eb80578b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:28.175796 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd316f-4377-4bcc-801a-c77eb80578b2 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:28.176150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd316f-4377-4bcc-801a-c77eb80578b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:28.176877 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcd316f-4377-4bcc-801a-c77eb80578b2 admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:28.177797 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcd316f-4377-4bcc-801a-c77eb80578b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:28.180266 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcd316f-4377-4bcc-801a-c77eb80578b2 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:28.224066 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc8a9e-dd99-4a19-96d1-0cd8c6d8b734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:28.224706 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc8a9e-dd99-4a19-96d1-0cd8c6d8b734 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:28.224987 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc8a9e-dd99-4a19-96d1-0cd8c6d8b734 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:28.225291 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc8a9e-dd99-4a19-96d1-0cd8c6d8b734 admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:28.226011 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc8a9e-dd99-4a19-96d1-0cd8c6d8b734 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:28.228018 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc8a9e-dd99-4a19-96d1-0cd8c6d8b734 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:28.273061 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 911/1794] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:55:27 2022] POST /identity/v3/users => generated 285 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:55:28.277498 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd316f-4377-4bcc-801a-c77eb80578b2 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:28.302718 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcd316f-4377-4bcc-801a-c77eb80578b2 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:28.315201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0996cc5-6b12-4b34-954b-2f26396a07d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:28.319685 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0996cc5-6b12-4b34-954b-2f26396a07d1 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:28.319685 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0996cc5-6b12-4b34-954b-2f26396a07d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:28.319685 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0996cc5-6b12-4b34-954b-2f26396a07d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:28.319918 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 884/1795] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:55:27 2022] POST /identity/v3/users => generated 309 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:28.320694 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc8a9e-dd99-4a19-96d1-0cd8c6d8b734 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:28.325745 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0996cc5-6b12-4b34-954b-2f26396a07d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:28.327587 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0996cc5-6b12-4b34-954b-2f26396a07d1 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:28.338715 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc8a9e-dd99-4a19-96d1-0cd8c6d8b734 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:28.351643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9777207e-2da8-469c-87ea-9e4a16dc61a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:28.352386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777207e-2da8-469c-87ea-9e4a16dc61a7 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:28.352582 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777207e-2da8-469c-87ea-9e4a16dc61a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:28.352826 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9777207e-2da8-469c-87ea-9e4a16dc61a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:28.353524 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9777207e-2da8-469c-87ea-9e4a16dc61a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:28.355336 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9777207e-2da8-469c-87ea-9e4a16dc61a7 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:28.356610 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 912/1796] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:28 2022] GET /identity/v3/roles => generated 1427 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:55:28.360873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0996cc5-6b12-4b34-954b-2f26396a07d1 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:28.379365 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 885/1797] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:28 2022] GET /identity/v3/roles => generated 1427 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:55:28.385304 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9777207e-2da8-469c-87ea-9e4a16dc61a7 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:28.396495 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0996cc5-6b12-4b34-954b-2f26396a07d1 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:28.410578 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf6097-72e1-47c3-b53c-1ca9c852e2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:28.413444 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf6097-72e1-47c3-b53c-1ca9c852e2a5 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:28.413444 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf6097-72e1-47c3-b53c-1ca9c852e2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:28.413444 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf6097-72e1-47c3-b53c-1ca9c852e2a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:28.413444 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf6097-72e1-47c3-b53c-1ca9c852e2a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:28.415878 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9777207e-2da8-469c-87ea-9e4a16dc61a7 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:28.418213 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf6097-72e1-47c3-b53c-1ca9c852e2a5 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:28.427233 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb98c47-e5a4-42fb-bac4-9090d9c73274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:28.429329 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb98c47-e5a4-42fb-bac4-9090d9c73274 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:28.429329 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb98c47-e5a4-42fb-bac4-9090d9c73274 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:28.429642 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb98c47-e5a4-42fb-bac4-9090d9c73274 admin admin] PATH_INFO: `/v3/projects/ca176d06824d4b76ae982cc94c1f6a4b/users/98869d51ab974392b53973fe535ee6de/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:28.462483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 913/1798] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:28 2022] GET /identity/v3/roles => generated 1427 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:28.469345 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf6097-72e1-47c3-b53c-1ca9c852e2a5 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:28.480592 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb98c47-e5a4-42fb-bac4-9090d9c73274 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca176d06824d4b76ae982cc94c1f6a4b, user_id=98869d51ab974392b53973fe535ee6de, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:28.484088 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb98c47-e5a4-42fb-bac4-9090d9c73274 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:28.491583 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf6097-72e1-47c3-b53c-1ca9c852e2a5 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:28.504242 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd2951-a93b-4ee4-8637-cbc2da2bb33c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:28.504866 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd2951-a93b-4ee4-8637-cbc2da2bb33c admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:28.505161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd2951-a93b-4ee4-8637-cbc2da2bb33c admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:28.505445 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd2951-a93b-4ee4-8637-cbc2da2bb33c admin admin] PATH_INFO: `/v3/projects/3d32e15bfa92410bb1d89155c3b7aa85/users/0f59b356948c4ace8f26a6436051f50e/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:28.541071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd2951-a93b-4ee4-8637-cbc2da2bb33c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d32e15bfa92410bb1d89155c3b7aa85, user_id=0f59b356948c4ace8f26a6436051f50e, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:28.544519 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd2951-a93b-4ee4-8637-cbc2da2bb33c admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:28.587736 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:28.605272 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 886/1799] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:28 2022] PUT /identity/v3/projects/ca176d06824d4b76ae982cc94c1f6a4b/users/98869d51ab974392b53973fe535ee6de/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:28.609262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb98c47-e5a4-42fb-bac4-9090d9c73274 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:28.659753 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 914/1800] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:28 2022] PUT /identity/v3/projects/3d32e15bfa92410bb1d89155c3b7aa85/users/0f59b356948c4ace8f26a6436051f50e/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:28.661335 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075386c5-f913-4ace-9714-cef0d5bce5c1 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:28.662448 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075386c5-f913-4ace-9714-cef0d5bce5c1 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:28.662740 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075386c5-f913-4ace-9714-cef0d5bce5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:28.734702 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-075386c5-f913-4ace-9714-cef0d5bce5c1 None None] MFA Rules not processed for user `0f59b356948c4ace8f26a6436051f50e`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:28.736371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-075386c5-f913-4ace-9714-cef0d5bce5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:28.776838 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb98c47-e5a4-42fb-bac4-9090d9c73274 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:28.790256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:28.903122 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1868d01-46b3-4568-a721-4aacab747dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:28.903808 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1868d01-46b3-4568-a721-4aacab747dbe admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:28.904178 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1868d01-46b3-4568-a721-4aacab747dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:28.904668 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1868d01-46b3-4568-a721-4aacab747dbe admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:28.915370 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1868d01-46b3-4568-a721-4aacab747dbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:28.923884 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1868d01-46b3-4568-a721-4aacab747dbe admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:28.941110 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 915/1801] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:28 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:28.942729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075386c5-f913-4ace-9714-cef0d5bce5c1 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:28.958515 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 887/1802] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:28 2022] GET /identity/v3/roles => generated 1427 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:28.966390 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1868d01-46b3-4568-a721-4aacab747dbe admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:28.980125 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075386c5-f913-4ace-9714-cef0d5bce5c1 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:28.991623 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1868d01-46b3-4568-a721-4aacab747dbe admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:28.992906 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78496795-e89c-4328-93e9-13bf6c1f66f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:28.993629 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78496795-e89c-4328-93e9-13bf6c1f66f5 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:28.993910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78496795-e89c-4328-93e9-13bf6c1f66f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:28.994333 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78496795-e89c-4328-93e9-13bf6c1f66f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:29.008283 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78496795-e89c-4328-93e9-13bf6c1f66f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:29.010668 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78496795-e89c-4328-93e9-13bf6c1f66f5 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:29.012061 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbadb6d-6ec4-42db-bc12-651755091464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:29.012781 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbadb6d-6ec4-42db-bc12-651755091464 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:29.013901 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbadb6d-6ec4-42db-bc12-651755091464 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:29.014411 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbadb6d-6ec4-42db-bc12-651755091464 admin admin] PATH_INFO: `/v3/projects/ca176d06824d4b76ae982cc94c1f6a4b/users/98869d51ab974392b53973fe535ee6de/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:29.026564 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbadb6d-6ec4-42db-bc12-651755091464 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca176d06824d4b76ae982cc94c1f6a4b, user_id=98869d51ab974392b53973fe535ee6de, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:29.031322 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fbadb6d-6ec4-42db-bc12-651755091464 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:29.048269 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 888/1803] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:28 2022] PUT /identity/v3/projects/ca176d06824d4b76ae982cc94c1f6a4b/users/98869d51ab974392b53973fe535ee6de/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:29.066441 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d2409-463a-4532-804a-61051a60aa8d None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:29.066848 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d2409-463a-4532-804a-61051a60aa8d None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:29.067411 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d2409-463a-4532-804a-61051a60aa8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:29.171096 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-3e8d2409-463a-4532-804a-61051a60aa8d None None] MFA Rules not processed for user `98869d51ab974392b53973fe535ee6de`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:29.171096 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-3e8d2409-463a-4532-804a-61051a60aa8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:29.194444 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 916/1804] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:55:28 2022] POST /identity/v3/projects => generated 368 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:29.203397 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78496795-e89c-4328-93e9-13bf6c1f66f5 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:29.326433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 889/1805] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:29 2022] POST /identity/v3/auth/tokens => generated 3357 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:29.340360 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d2409-463a-4532-804a-61051a60aa8d None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:29.342608 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-3e8d2409-463a-4532-804a-61051a60aa8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:29.367036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:29.423164 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78496795-e89c-4328-93e9-13bf6c1f66f5 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:29.442522 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95245723-5931-4fc8-8057-4c5f217c8b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:29.443172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95245723-5931-4fc8-8057-4c5f217c8b1f admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:29.443483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95245723-5931-4fc8-8057-4c5f217c8b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:29.443764 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95245723-5931-4fc8-8057-4c5f217c8b1f admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:29.444422 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95245723-5931-4fc8-8057-4c5f217c8b1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:29.447442 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95245723-5931-4fc8-8057-4c5f217c8b1f admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:29.463204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d2409-463a-4532-804a-61051a60aa8d None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:29.623167 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 917/1806] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:55:29 2022] POST /identity/v3/users => generated 291 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:29.628385 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fd5b1-6a84-48cf-81b2-0729334eefd9 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:29.632495 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fd5b1-6a84-48cf-81b2-0729334eefd9 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:29.632495 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5fd5b1-6a84-48cf-81b2-0729334eefd9 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:29.637390 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 918/1807] 10.222.0.115 () {54 vars in 926 bytes} [Tue Aug 2 10:55:29 2022] GET /identity => generated 271 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:55:29.638849 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5fd5b1-6a84-48cf-81b2-0729334eefd9 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:29.646008 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-2f5fd5b1-6a84-48cf-81b2-0729334eefd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:29.650002 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c413aaa-b0a7-46f5-9c3c-4f30d733fbfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:29.652347 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c413aaa-b0a7-46f5-9c3c-4f30d733fbfe service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:29.652876 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c413aaa-b0a7-46f5-9c3c-4f30d733fbfe service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:29.654674 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c413aaa-b0a7-46f5-9c3c-4f30d733fbfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:29.694611 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5fd5b1-6a84-48cf-81b2-0729334eefd9 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:29.706459 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800ea8e-ad9f-48ec-9a70-fcc2d5d5c99e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:29.707139 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800ea8e-ad9f-48ec-9a70-fcc2d5d5c99e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:29.707429 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800ea8e-ad9f-48ec-9a70-fcc2d5d5c99e admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:29.707706 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800ea8e-ad9f-48ec-9a70-fcc2d5d5c99e admin admin] PATH_INFO: `/v3/projects/f4a82fa22d1f4773a0bafe9a0ef7e951` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:29.711935 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800ea8e-ad9f-48ec-9a70-fcc2d5d5c99e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4a82fa22d1f4773a0bafe9a0ef7e951)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:29.713889 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800ea8e-ad9f-48ec-9a70-fcc2d5d5c99e admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:29.776280 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c413aaa-b0a7-46f5-9c3c-4f30d733fbfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:29.777853 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c413aaa-b0a7-46f5-9c3c-4f30d733fbfe service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:29.794361 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 890/1808] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:29 2022] GET /identity/v3/auth/tokens => generated 3357 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:55:29.795470 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c413aaa-b0a7-46f5-9c3c-4f30d733fbfe service cinder] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:29.837498 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c413aaa-b0a7-46f5-9c3c-4f30d733fbfe service cinder] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:29.859526 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d93305-d082-412a-a395-1d5238d956ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:29.862239 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d93305-d082-412a-a395-1d5238d956ff admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:29.871616 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-2800ea8e-ad9f-48ec-9a70-fcc2d5d5c99e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4a82fa22d1f4773a0bafe9a0ef7e951', 'request_id': 'req-2800ea8e-ad9f-48ec-9a70-fcc2d5d5c99e'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:29.875838 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d93305-d082-412a-a395-1d5238d956ff admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:29.891083 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d93305-d082-412a-a395-1d5238d956ff admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:29.891083 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d93305-d082-412a-a395-1d5238d956ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:29.891083 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d93305-d082-412a-a395-1d5238d956ff admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:29.917605 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-2800ea8e-ad9f-48ec-9a70-fcc2d5d5c99e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4a82fa22d1f4773a0bafe9a0ef7e951', 'request_id': 'req-2800ea8e-ad9f-48ec-9a70-fcc2d5d5c99e'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:29.947795 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 891/1809] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:29 2022] GET /identity/v3/roles => generated 1427 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:29.952361 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af182d48-013c-47de-8b03-03e2f6ddd00e None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:29.952361 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af182d48-013c-47de-8b03-03e2f6ddd00e None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:29.952361 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af182d48-013c-47de-8b03-03e2f6ddd00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:30.033845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-af182d48-013c-47de-8b03-03e2f6ddd00e None None] MFA Rules not processed for user `d0478bd540f446ec810e2091c46ac49c`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:30.035035 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-af182d48-013c-47de-8b03-03e2f6ddd00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:30.037786 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-2800ea8e-ad9f-48ec-9a70-fcc2d5d5c99e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4a82fa22d1f4773a0bafe9a0ef7e951', 'request_id': 'req-2800ea8e-ad9f-48ec-9a70-fcc2d5d5c99e'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:30.067964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-2800ea8e-ad9f-48ec-9a70-fcc2d5d5c99e admin admin] The token cache is being invalidate because project f4a82fa22d1f4773a0bafe9a0ef7e951 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:55:30.068624 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-2800ea8e-ad9f-48ec-9a70-fcc2d5d5c99e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:30.070234 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-2800ea8e-ad9f-48ec-9a70-fcc2d5d5c99e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:30.072305 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 919/1810] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:55:29 2022] DELETE /identity/v3/projects/f4a82fa22d1f4773a0bafe9a0ef7e951 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:30.073434 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800ea8e-ad9f-48ec-9a70-fcc2d5d5c99e admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:30.075245 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-2800ea8e-ad9f-48ec-9a70-fcc2d5d5c99e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:30.090857 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:30.206367 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800ea8e-ad9f-48ec-9a70-fcc2d5d5c99e admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:30.212783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:30.218551 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5850b-6a90-4515-9728-616b012f8c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:30.220523 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5850b-6a90-4515-9728-616b012f8c6e admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:30.220805 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5850b-6a90-4515-9728-616b012f8c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:30.221090 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d5850b-6a90-4515-9728-616b012f8c6e admin admin] PATH_INFO: `/v3/projects/171c9eabe04846809b6d2696aa09085d/users/56d64a36a58e4ebb88e7f38d448b8da9/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:30.257302 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 892/1811] 10.222.0.115 () {58 vars in 1023 bytes} [Tue Aug 2 10:55:29 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:30.258923 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af182d48-013c-47de-8b03-03e2f6ddd00e None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:30.261521 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-af182d48-013c-47de-8b03-03e2f6ddd00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:30.288415 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5850b-6a90-4515-9728-616b012f8c6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=171c9eabe04846809b6d2696aa09085d, user_id=56d64a36a58e4ebb88e7f38d448b8da9, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:30.299018 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d5850b-6a90-4515-9728-616b012f8c6e admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:30.308642 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af182d48-013c-47de-8b03-03e2f6ddd00e None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:30.317140 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18306938-1da5-4496-95c6-c8b22615961b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:30.317884 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18306938-1da5-4496-95c6-c8b22615961b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:30.318165 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18306938-1da5-4496-95c6-c8b22615961b admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:30.318536 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18306938-1da5-4496-95c6-c8b22615961b admin admin] PATH_INFO: `/v3/users/f864ba3794464d0cbcd200f6d140aa5c` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:30.319808 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18306938-1da5-4496-95c6-c8b22615961b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f864ba3794464d0cbcd200f6d140aa5c)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:30.321843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18306938-1da5-4496-95c6-c8b22615961b admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:30.396576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 920/1812] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:30 2022] PUT /identity/v3/projects/171c9eabe04846809b6d2696aa09085d/users/56d64a36a58e4ebb88e7f38d448b8da9/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:30.397656 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5850b-6a90-4515-9728-616b012f8c6e admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:30.413832 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-57d5850b-6a90-4515-9728-616b012f8c6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:30.521345 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d5850b-6a90-4515-9728-616b012f8c6e admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:30.528481 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804116e-075b-4637-bd24-91501a18e371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:30.529276 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804116e-075b-4637-bd24-91501a18e371 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:30.530090 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804116e-075b-4637-bd24-91501a18e371 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:30.530090 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7804116e-075b-4637-bd24-91501a18e371 admin admin] PATH_INFO: `/v3/users/78a9147f3daf4c64a9261c4297b3ff9c` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:30.531733 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804116e-075b-4637-bd24-91501a18e371 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78a9147f3daf4c64a9261c4297b3ff9c)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:30.534321 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7804116e-075b-4637-bd24-91501a18e371 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:30.613577 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-18306938-1da5-4496-95c6-c8b22615961b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f864ba3794464d0cbcd200f6d140aa5c'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:30.623441 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-18306938-1da5-4496-95c6-c8b22615961b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f864ba3794464d0cbcd200f6d140aa5c'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:30.624902 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-18306938-1da5-4496-95c6-c8b22615961b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f864ba3794464d0cbcd200f6d140aa5c'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:30.739196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-18306938-1da5-4496-95c6-c8b22615961b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f864ba3794464d0cbcd200f6d140aa5c'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:30.765235 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-18306938-1da5-4496-95c6-c8b22615961b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f864ba3794464d0cbcd200f6d140aa5c'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:30.779916 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 893/1813] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:55:30 2022] DELETE /identity/v3/users/f864ba3794464d0cbcd200f6d140aa5c => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:30.780961 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18306938-1da5-4496-95c6-c8b22615961b admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:30.786790 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-18306938-1da5-4496-95c6-c8b22615961b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:30.845389 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-7804116e-075b-4637-bd24-91501a18e371 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78a9147f3daf4c64a9261c4297b3ff9c'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:30.846994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-7804116e-075b-4637-bd24-91501a18e371 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78a9147f3daf4c64a9261c4297b3ff9c'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:30.915893 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-7804116e-075b-4637-bd24-91501a18e371 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78a9147f3daf4c64a9261c4297b3ff9c'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:30.927563 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-7804116e-075b-4637-bd24-91501a18e371 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78a9147f3daf4c64a9261c4297b3ff9c'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:30.962253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18306938-1da5-4496-95c6-c8b22615961b admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:30.968068 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-7804116e-075b-4637-bd24-91501a18e371 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78a9147f3daf4c64a9261c4297b3ff9c'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:30.979641 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6486a1-7266-4a91-927a-557005d0f35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:30.979845 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 921/1814] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:55:30 2022] DELETE /identity/v3/users/78a9147f3daf4c64a9261c4297b3ff9c => generated 0 bytes in 582 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:30.980625 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804116e-075b-4637-bd24-91501a18e371 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:30.982585 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6486a1-7266-4a91-927a-557005d0f35c admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:30.982978 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6486a1-7266-4a91-927a-557005d0f35c admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:30.983364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6486a1-7266-4a91-927a-557005d0f35c admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:30.984106 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6486a1-7266-4a91-927a-557005d0f35c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:30.984570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-7804116e-075b-4637-bd24-91501a18e371 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:30.985754 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6486a1-7266-4a91-927a-557005d0f35c admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:31.014617 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 894/1815] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:30 2022] GET /identity/v3/roles => generated 1427 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:31.016235 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4575ca-9c5d-4176-b7a7-193fb5872594 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:31.016867 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4575ca-9c5d-4176-b7a7-193fb5872594 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:31.016867 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4575ca-9c5d-4176-b7a7-193fb5872594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:31.021877 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:31.061331 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-cb4575ca-9c5d-4176-b7a7-193fb5872594 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:31.067356 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-cb4575ca-9c5d-4176-b7a7-193fb5872594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:31.186591 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7804116e-075b-4637-bd24-91501a18e371 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:31.195016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ea3e3b-dcaf-4e36-bc36-d050a7133f0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:31.198222 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 895/1816] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:31 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:31.201805 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4575ca-9c5d-4176-b7a7-193fb5872594 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:31.205195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea3e3b-dcaf-4e36-bc36-d050a7133f0e service nova] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:31.205195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea3e3b-dcaf-4e36-bc36-d050a7133f0e service nova] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:31.205195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ea3e3b-dcaf-4e36-bc36-d050a7133f0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:31.206013 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-cb4575ca-9c5d-4176-b7a7-193fb5872594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:31.207449 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-75ea3e3b-dcaf-4e36-bc36-d050a7133f0e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:31.224928 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4575ca-9c5d-4176-b7a7-193fb5872594 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:31.239017 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb68a0-4dcf-4c64-ae16-b0a30c632e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:31.239017 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb68a0-4dcf-4c64-ae16-b0a30c632e8a admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:31.239503 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ea3e3b-dcaf-4e36-bc36-d050a7133f0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:31.239609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb68a0-4dcf-4c64-ae16-b0a30c632e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:31.239609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fb68a0-4dcf-4c64-ae16-b0a30c632e8a admin admin] PATH_INFO: `/v3/projects/171c9eabe04846809b6d2696aa09085d/users/56d64a36a58e4ebb88e7f38d448b8da9/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:31.239735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ea3e3b-dcaf-4e36-bc36-d050a7133f0e service nova] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:31.239801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fb68a0-4dcf-4c64-ae16-b0a30c632e8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=171c9eabe04846809b6d2696aa09085d, user_id=56d64a36a58e4ebb88e7f38d448b8da9, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:31.239801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fb68a0-4dcf-4c64-ae16-b0a30c632e8a admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:31.244865 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:31.265816 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 896/1817] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:31 2022] PUT /identity/v3/projects/171c9eabe04846809b6d2696aa09085d/users/56d64a36a58e4ebb88e7f38d448b8da9/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:31.265816 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb68a0-4dcf-4c64-ae16-b0a30c632e8a admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:31.267492 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 922/1818] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:30 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:31.274216 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d9b37-d99b-4ac7-84b8-c8f3519620a9 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:31.274424 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-41fb68a0-4dcf-4c64-ae16-b0a30c632e8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:31.274509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d9b37-d99b-4ac7-84b8-c8f3519620a9 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:31.274770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6d9b37-d99b-4ac7-84b8-c8f3519620a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:31.356065 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-fc6d9b37-d99b-4ac7-84b8-c8f3519620a9 None None] MFA Rules not processed for user `56d64a36a58e4ebb88e7f38d448b8da9`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:31.358408 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-fc6d9b37-d99b-4ac7-84b8-c8f3519620a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:31.359185 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fb68a0-4dcf-4c64-ae16-b0a30c632e8a admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:31.363904 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad0995-ccc3-4f07-95ee-86db3e8f28b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:31.364999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad0995-ccc3-4f07-95ee-86db3e8f28b2 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:31.366314 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad0995-ccc3-4f07-95ee-86db3e8f28b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:31.366917 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ad0995-ccc3-4f07-95ee-86db3e8f28b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:31.502843 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 923/1819] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:31 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:31.515256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d9b37-d99b-4ac7-84b8-c8f3519620a9 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:31.518923 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-fc6d9b37-d99b-4ac7-84b8-c8f3519620a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:31.546337 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad0995-ccc3-4f07-95ee-86db3e8f28b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:31.549136 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ad0995-ccc3-4f07-95ee-86db3e8f28b2 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:31.591524 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 897/1820] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:31 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:31.723013 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6d9b37-d99b-4ac7-84b8-c8f3519620a9 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:31.728217 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99eec24-e4c8-4ac1-8864-830b920c9a14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:31.728870 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99eec24-e4c8-4ac1-8864-830b920c9a14 service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:31.729156 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99eec24-e4c8-4ac1-8864-830b920c9a14 service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:31.729434 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99eec24-e4c8-4ac1-8864-830b920c9a14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:31.759817 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99eec24-e4c8-4ac1-8864-830b920c9a14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:31.762682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99eec24-e4c8-4ac1-8864-830b920c9a14 service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:31.792520 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 924/1821] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:31 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:31.915184 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad0995-ccc3-4f07-95ee-86db3e8f28b2 service neutron] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:31.918398 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-61ad0995-ccc3-4f07-95ee-86db3e8f28b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:31.976920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ad0995-ccc3-4f07-95ee-86db3e8f28b2 service neutron] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:31.996464 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b88fd-fa25-4c01-a662-b709c3656687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:31.996464 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b88fd-fa25-4c01-a662-b709c3656687 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:31.996464 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b88fd-fa25-4c01-a662-b709c3656687 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:31.996464 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b88fd-fa25-4c01-a662-b709c3656687 admin admin] PATH_INFO: `/v3/users/995b34126c9f42eeb16aa7bc6ba3d912` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:31.998164 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b88fd-fa25-4c01-a662-b709c3656687 admin admin] RBAC: Authorizing `identity:delete_user(user_id=995b34126c9f42eeb16aa7bc6ba3d912)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:32.001316 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b88fd-fa25-4c01-a662-b709c3656687 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:32.082764 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99eec24-e4c8-4ac1-8864-830b920c9a14 service cinder] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:32.086230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-d99eec24-e4c8-4ac1-8864-830b920c9a14 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:32.114575 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99eec24-e4c8-4ac1-8864-830b920c9a14 service cinder] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:32.126480 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47320d-9ff8-472d-80b2-8236eb98754a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:32.126480 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47320d-9ff8-472d-80b2-8236eb98754a service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:32.126480 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47320d-9ff8-472d-80b2-8236eb98754a service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:32.126480 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47320d-9ff8-472d-80b2-8236eb98754a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:32.126768 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-6c47320d-9ff8-472d-80b2-8236eb98754a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:32.170443 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:32.240593 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-762b88fd-fa25-4c01-a662-b709c3656687 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '995b34126c9f42eeb16aa7bc6ba3d912'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:32.256709 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-762b88fd-fa25-4c01-a662-b709c3656687 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '995b34126c9f42eeb16aa7bc6ba3d912'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:32.259170 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-762b88fd-fa25-4c01-a662-b709c3656687 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '995b34126c9f42eeb16aa7bc6ba3d912'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:32.283109 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47320d-9ff8-472d-80b2-8236eb98754a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:32.284902 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47320d-9ff8-472d-80b2-8236eb98754a service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:32.297378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 925/1822] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:32 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:32.336706 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-762b88fd-fa25-4c01-a662-b709c3656687 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '995b34126c9f42eeb16aa7bc6ba3d912'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:32.359788 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-762b88fd-fa25-4c01-a662-b709c3656687 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '995b34126c9f42eeb16aa7bc6ba3d912'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:32.379221 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 898/1823] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:55:31 2022] DELETE /identity/v3/users/995b34126c9f42eeb16aa7bc6ba3d912 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:32.381904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47cdf5-7921-47da-b0a6-45fdd3d3be32 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:32.382206 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47cdf5-7921-47da-b0a6-45fdd3d3be32 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:32.382490 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd47cdf5-7921-47da-b0a6-45fdd3d3be32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:32.454474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-bd47cdf5-7921-47da-b0a6-45fdd3d3be32 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:32.457458 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-bd47cdf5-7921-47da-b0a6-45fdd3d3be32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:32.598174 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 926/1824] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:32 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:32.767958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b88fd-fa25-4c01-a662-b709c3656687 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:32.769604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-762b88fd-fa25-4c01-a662-b709c3656687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:32.812635 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:32.921115 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b88fd-fa25-4c01-a662-b709c3656687 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:33.016945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0e60f-61c6-4a9c-858f-d75d3e6b00df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:33.016945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0e60f-61c6-4a9c-858f-d75d3e6b00df service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:33.016945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0e60f-61c6-4a9c-858f-d75d3e6b00df service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:33.016945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0e60f-61c6-4a9c-858f-d75d3e6b00df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:33.017982 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc0e60f-61c6-4a9c-858f-d75d3e6b00df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:33.017982 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc0e60f-61c6-4a9c-858f-d75d3e6b00df service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:33.017982 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 899/1825] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:32 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:33.169354 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47cdf5-7921-47da-b0a6-45fdd3d3be32 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:33.171069 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-bd47cdf5-7921-47da-b0a6-45fdd3d3be32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:33.183830 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd47cdf5-7921-47da-b0a6-45fdd3d3be32 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:33.189612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4173903-ab9d-4e95-971c-0cd10bd0088a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:33.190359 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4173903-ab9d-4e95-971c-0cd10bd0088a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:33.190718 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4173903-ab9d-4e95-971c-0cd10bd0088a admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:33.191118 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4173903-ab9d-4e95-971c-0cd10bd0088a admin admin] PATH_INFO: `/v3/projects/d3f3a751b6dc4a6aa51a204a1eadc7f0` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:33.192449 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4173903-ab9d-4e95-971c-0cd10bd0088a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3f3a751b6dc4a6aa51a204a1eadc7f0)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:33.197351 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4173903-ab9d-4e95-971c-0cd10bd0088a admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:33.315038 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0e60f-61c6-4a9c-858f-d75d3e6b00df service neutron] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:33.316839 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-4fc0e60f-61c6-4a9c-858f-d75d3e6b00df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:33.366427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0e60f-61c6-4a9c-858f-d75d3e6b00df service neutron] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:33.404431 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e4173903-ab9d-4e95-971c-0cd10bd0088a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3f3a751b6dc4a6aa51a204a1eadc7f0', 'request_id': 'req-e4173903-ab9d-4e95-971c-0cd10bd0088a'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:33.438163 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e4173903-ab9d-4e95-971c-0cd10bd0088a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3f3a751b6dc4a6aa51a204a1eadc7f0', 'request_id': 'req-e4173903-ab9d-4e95-971c-0cd10bd0088a'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:33.550412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e4173903-ab9d-4e95-971c-0cd10bd0088a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3f3a751b6dc4a6aa51a204a1eadc7f0', 'request_id': 'req-e4173903-ab9d-4e95-971c-0cd10bd0088a'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:33.566622 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e4173903-ab9d-4e95-971c-0cd10bd0088a admin admin] The token cache is being invalidate because project d3f3a751b6dc4a6aa51a204a1eadc7f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:55:33.566906 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e4173903-ab9d-4e95-971c-0cd10bd0088a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:33.576426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e4173903-ab9d-4e95-971c-0cd10bd0088a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:33.579812 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 927/1826] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:55:33 2022] DELETE /identity/v3/projects/d3f3a751b6dc4a6aa51a204a1eadc7f0 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:33.588428 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4173903-ab9d-4e95-971c-0cd10bd0088a admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:33.590647 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-e4173903-ab9d-4e95-971c-0cd10bd0088a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:33.597630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3becd44-8ae8-45ed-b75f-30354eeff69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:33.598379 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3becd44-8ae8-45ed-b75f-30354eeff69b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:33.598668 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3becd44-8ae8-45ed-b75f-30354eeff69b admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:33.598952 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3becd44-8ae8-45ed-b75f-30354eeff69b admin admin] PATH_INFO: `/v3/projects/85e2e21340eb4754be5a6bbe2e56905a` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:33.600472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3becd44-8ae8-45ed-b75f-30354eeff69b admin admin] RBAC: Authorizing `identity:delete_project(project_id=85e2e21340eb4754be5a6bbe2e56905a)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:33.604442 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3becd44-8ae8-45ed-b75f-30354eeff69b admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:33.618134 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4173903-ab9d-4e95-971c-0cd10bd0088a admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:33.624845 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c7e51-cd6c-48f5-8bc3-c1d4f06214cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:33.625890 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c7e51-cd6c-48f5-8bc3-c1d4f06214cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:33.626402 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c7e51-cd6c-48f5-8bc3-c1d4f06214cf admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:33.626894 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235c7e51-cd6c-48f5-8bc3-c1d4f06214cf admin admin] PATH_INFO: `/v3/users/98869d51ab974392b53973fe535ee6de` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:33.628707 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235c7e51-cd6c-48f5-8bc3-c1d4f06214cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=98869d51ab974392b53973fe535ee6de)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:33.631587 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235c7e51-cd6c-48f5-8bc3-c1d4f06214cf admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:33.771718 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-c3becd44-8ae8-45ed-b75f-30354eeff69b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85e2e21340eb4754be5a6bbe2e56905a', 'request_id': 'req-c3becd44-8ae8-45ed-b75f-30354eeff69b'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:33.853328 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-c3becd44-8ae8-45ed-b75f-30354eeff69b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85e2e21340eb4754be5a6bbe2e56905a', 'request_id': 'req-c3becd44-8ae8-45ed-b75f-30354eeff69b'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:33.859923 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-c3becd44-8ae8-45ed-b75f-30354eeff69b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85e2e21340eb4754be5a6bbe2e56905a', 'request_id': 'req-c3becd44-8ae8-45ed-b75f-30354eeff69b'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:33.872946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-c3becd44-8ae8-45ed-b75f-30354eeff69b admin admin] The token cache is being invalidate because project 85e2e21340eb4754be5a6bbe2e56905a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:55:33.873427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-c3becd44-8ae8-45ed-b75f-30354eeff69b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:33.874143 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-c3becd44-8ae8-45ed-b75f-30354eeff69b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:33.877019 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 900/1827] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:55:33 2022] DELETE /identity/v3/projects/85e2e21340eb4754be5a6bbe2e56905a => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:55:33.883082 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15753f37-fa15-4c4e-b8b4-210000d40631 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:33.885266 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15753f37-fa15-4c4e-b8b4-210000d40631 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:33.885266 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15753f37-fa15-4c4e-b8b4-210000d40631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:33.898613 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:33.905966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-235c7e51-cd6c-48f5-8bc3-c1d4f06214cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98869d51ab974392b53973fe535ee6de'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:33.907419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-235c7e51-cd6c-48f5-8bc3-c1d4f06214cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98869d51ab974392b53973fe535ee6de'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:33.912983 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-15753f37-fa15-4c4e-b8b4-210000d40631 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:33.914663 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-15753f37-fa15-4c4e-b8b4-210000d40631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:33.968799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-235c7e51-cd6c-48f5-8bc3-c1d4f06214cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98869d51ab974392b53973fe535ee6de'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:33.973776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:34.002406 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-235c7e51-cd6c-48f5-8bc3-c1d4f06214cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98869d51ab974392b53973fe535ee6de'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:34.010574 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 901/1828] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:33 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:34.013439 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762459e1-2bb8-4dc5-81ed-5ad4317dae84 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:34.013716 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762459e1-2bb8-4dc5-81ed-5ad4317dae84 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:34.014036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762459e1-2bb8-4dc5-81ed-5ad4317dae84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:34.020866 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-235c7e51-cd6c-48f5-8bc3-c1d4f06214cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98869d51ab974392b53973fe535ee6de'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:34.035796 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 928/1829] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:55:33 2022] DELETE /identity/v3/users/98869d51ab974392b53973fe535ee6de => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:34.052033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-762459e1-2bb8-4dc5-81ed-5ad4317dae84 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:34.056277 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-762459e1-2bb8-4dc5-81ed-5ad4317dae84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:34.123373 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 902/1830] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:34 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:34.127692 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c7e51-cd6c-48f5-8bc3-c1d4f06214cf admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:34.134780 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235c7e51-cd6c-48f5-8bc3-c1d4f06214cf admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:34.140287 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677b69e-1514-43e4-92ee-822b0dbaae2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:34.140916 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677b69e-1514-43e4-92ee-822b0dbaae2e admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:34.141207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677b69e-1514-43e4-92ee-822b0dbaae2e admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:34.141486 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677b69e-1514-43e4-92ee-822b0dbaae2e admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:34.142302 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677b69e-1514-43e4-92ee-822b0dbaae2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:55:34.142725 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677b69e-1514-43e4-92ee-822b0dbaae2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:34.144262 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677b69e-1514-43e4-92ee-822b0dbaae2e admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:34.156254 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 929/1831] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:55:34 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:55:34.160817 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762459e1-2bb8-4dc5-81ed-5ad4317dae84 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:34.170725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762459e1-2bb8-4dc5-81ed-5ad4317dae84 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:34.175411 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8065bf12-8b8c-427d-87be-2dc40e7ea0ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:34.176014 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065bf12-8b8c-427d-87be-2dc40e7ea0ad admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:34.176289 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065bf12-8b8c-427d-87be-2dc40e7ea0ad admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:34.176559 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8065bf12-8b8c-427d-87be-2dc40e7ea0ad admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:34.177265 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8065bf12-8b8c-427d-87be-2dc40e7ea0ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:34.179187 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8065bf12-8b8c-427d-87be-2dc40e7ea0ad admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:34.273530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677b69e-1514-43e4-92ee-822b0dbaae2e admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:34.276505 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-e677b69e-1514-43e4-92ee-822b0dbaae2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:34.297217 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677b69e-1514-43e4-92ee-822b0dbaae2e admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:34.302955 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871d7c4-c05c-4c64-80f5-dab8f23704e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:34.303468 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 903/1832] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:55:34 2022] POST /identity/v3/projects => generated 394 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:34.303764 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871d7c4-c05c-4c64-80f5-dab8f23704e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:34.304048 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871d7c4-c05c-4c64-80f5-dab8f23704e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:34.304324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e871d7c4-c05c-4c64-80f5-dab8f23704e5 admin admin] PATH_INFO: `/v3/projects/2a013a5b22bd481b8967bc1d565de2ef` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:34.305602 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871d7c4-c05c-4c64-80f5-dab8f23704e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a013a5b22bd481b8967bc1d565de2ef)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:34.307394 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8065bf12-8b8c-427d-87be-2dc40e7ea0ad admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:34.311239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e871d7c4-c05c-4c64-80f5-dab8f23704e5 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:34.392084 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8065bf12-8b8c-427d-87be-2dc40e7ea0ad admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:34.400988 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5cb1c6-b88f-4a9f-ad16-a08998c930ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:34.400988 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5cb1c6-b88f-4a9f-ad16-a08998c930ba admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:34.400988 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5cb1c6-b88f-4a9f-ad16-a08998c930ba admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:34.400988 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5cb1c6-b88f-4a9f-ad16-a08998c930ba admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:34.401275 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5cb1c6-b88f-4a9f-ad16-a08998c930ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:34.401275 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5cb1c6-b88f-4a9f-ad16-a08998c930ba admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:34.447018 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e871d7c4-c05c-4c64-80f5-dab8f23704e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a013a5b22bd481b8967bc1d565de2ef', 'request_id': 'req-e871d7c4-c05c-4c64-80f5-dab8f23704e5'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:34.461210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e871d7c4-c05c-4c64-80f5-dab8f23704e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a013a5b22bd481b8967bc1d565de2ef', 'request_id': 'req-e871d7c4-c05c-4c64-80f5-dab8f23704e5'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:34.508830 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 904/1833] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:55:34 2022] POST /identity/v3/users => generated 298 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:55:34.511856 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5cb1c6-b88f-4a9f-ad16-a08998c930ba admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:34.538677 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5cb1c6-b88f-4a9f-ad16-a08998c930ba admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:34.578159 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e871d7c4-c05c-4c64-80f5-dab8f23704e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a013a5b22bd481b8967bc1d565de2ef', 'request_id': 'req-e871d7c4-c05c-4c64-80f5-dab8f23704e5'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:34.597225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e871d7c4-c05c-4c64-80f5-dab8f23704e5 admin admin] The token cache is being invalidate because project 2a013a5b22bd481b8967bc1d565de2ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:55:34.597463 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e871d7c4-c05c-4c64-80f5-dab8f23704e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:34.601201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e871d7c4-c05c-4c64-80f5-dab8f23704e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:34.604072 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 930/1834] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:55:34 2022] DELETE /identity/v3/projects/2a013a5b22bd481b8967bc1d565de2ef => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:55:34.609455 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871d7c4-c05c-4c64-80f5-dab8f23704e5 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:34.611221 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-e871d7c4-c05c-4c64-80f5-dab8f23704e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:34.684753 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff65ddd0-c2a0-4b76-84e4-f5610b7fd0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:34.687292 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65ddd0-c2a0-4b76-84e4-f5610b7fd0b5 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:34.687292 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65ddd0-c2a0-4b76-84e4-f5610b7fd0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:34.687292 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65ddd0-c2a0-4b76-84e4-f5610b7fd0b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:34.690085 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff65ddd0-c2a0-4b76-84e4-f5610b7fd0b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:34.692624 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff65ddd0-c2a0-4b76-84e4-f5610b7fd0b5 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:34.706776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 905/1835] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:34 2022] GET /identity/v3/roles => generated 1427 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:55:34.710302 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff65ddd0-c2a0-4b76-84e4-f5610b7fd0b5 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:34.712164 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-ff65ddd0-c2a0-4b76-84e4-f5610b7fd0b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:34.736768 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff65ddd0-c2a0-4b76-84e4-f5610b7fd0b5 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:34.737979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e871d7c4-c05c-4c64-80f5-dab8f23704e5 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:34.739061 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698e8b0-d7f7-4b65-b2df-2f01c55b40ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:34.740641 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698e8b0-d7f7-4b65-b2df-2f01c55b40ab admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:34.740641 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698e8b0-d7f7-4b65-b2df-2f01c55b40ab admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:34.740854 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c698e8b0-d7f7-4b65-b2df-2f01c55b40ab admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:34.741712 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698e8b0-d7f7-4b65-b2df-2f01c55b40ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:34.743317 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa32f36-12fe-45a6-a160-055db31ee4c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:34.743915 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c698e8b0-d7f7-4b65-b2df-2f01c55b40ab admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:34.744074 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa32f36-12fe-45a6-a160-055db31ee4c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:34.744560 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa32f36-12fe-45a6-a160-055db31ee4c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:34.745113 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa32f36-12fe-45a6-a160-055db31ee4c8 admin admin] PATH_INFO: `/v3/users/f2aa724a18564533aaf7b56d4aaa64c3` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:34.746395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa32f36-12fe-45a6-a160-055db31ee4c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2aa724a18564533aaf7b56d4aaa64c3)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:34.749058 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa32f36-12fe-45a6-a160-055db31ee4c8 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:34.777101 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 906/1836] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:34 2022] GET /identity/v3/roles => generated 1427 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:55:34.781632 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698e8b0-d7f7-4b65-b2df-2f01c55b40ab admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:34.793249 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c698e8b0-d7f7-4b65-b2df-2f01c55b40ab admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:34.801196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7a5ad-4820-4ec2-89a0-4c3bf4bb8427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:34.802074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7a5ad-4820-4ec2-89a0-4c3bf4bb8427 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:34.802419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7a5ad-4820-4ec2-89a0-4c3bf4bb8427 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:34.802767 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b7a5ad-4820-4ec2-89a0-4c3bf4bb8427 admin admin] PATH_INFO: `/v3/projects/a5e86f7861604d8c93f0737c4303f279/users/a6ac0e65fe8d4ba7aeedc6b424379a7c/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:34.823837 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7a5ad-4820-4ec2-89a0-4c3bf4bb8427 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a5e86f7861604d8c93f0737c4303f279, user_id=a6ac0e65fe8d4ba7aeedc6b424379a7c, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:34.829319 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b7a5ad-4820-4ec2-89a0-4c3bf4bb8427 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:34.962502 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 907/1837] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:34 2022] PUT /identity/v3/projects/a5e86f7861604d8c93f0737c4303f279/users/a6ac0e65fe8d4ba7aeedc6b424379a7c/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:34.964289 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7a5ad-4820-4ec2-89a0-4c3bf4bb8427 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:34.971023 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-64b7a5ad-4820-4ec2-89a0-4c3bf4bb8427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:34.987483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-baa32f36-12fe-45a6-a160-055db31ee4c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2aa724a18564533aaf7b56d4aaa64c3'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:34.987989 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-baa32f36-12fe-45a6-a160-055db31ee4c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2aa724a18564533aaf7b56d4aaa64c3'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:35.053697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b7a5ad-4820-4ec2-89a0-4c3bf4bb8427 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:35.058302 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf5916-8a2e-40e8-a7e9-2080441441ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:35.059071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf5916-8a2e-40e8-a7e9-2080441441ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:35.059360 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf5916-8a2e-40e8-a7e9-2080441441ef admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:35.060286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf5916-8a2e-40e8-a7e9-2080441441ef admin admin] PATH_INFO: `/v3/projects/ca176d06824d4b76ae982cc94c1f6a4b` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:35.061819 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf5916-8a2e-40e8-a7e9-2080441441ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca176d06824d4b76ae982cc94c1f6a4b)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:35.063827 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf5916-8a2e-40e8-a7e9-2080441441ef admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:35.088519 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-baa32f36-12fe-45a6-a160-055db31ee4c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2aa724a18564533aaf7b56d4aaa64c3'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:35.088519 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-baa32f36-12fe-45a6-a160-055db31ee4c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2aa724a18564533aaf7b56d4aaa64c3'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:35.100721 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-baa32f36-12fe-45a6-a160-055db31ee4c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2aa724a18564533aaf7b56d4aaa64c3'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:35.107363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 931/1838] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:55:34 2022] DELETE /identity/v3/users/f2aa724a18564533aaf7b56d4aaa64c3 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:35.108298 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a27d5-f00e-49e7-a167-b65ffb3d0b1f None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:35.108553 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a27d5-f00e-49e7-a167-b65ffb3d0b1f None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:35.108890 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7a27d5-f00e-49e7-a167-b65ffb3d0b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:35.140592 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-da7a27d5-f00e-49e7-a167-b65ffb3d0b1f None None] MFA Rules not processed for user `a6ac0e65fe8d4ba7aeedc6b424379a7c`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:35.141966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-da7a27d5-f00e-49e7-a167-b65ffb3d0b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:35.177886 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-7cdf5916-8a2e-40e8-a7e9-2080441441ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca176d06824d4b76ae982cc94c1f6a4b', 'request_id': 'req-7cdf5916-8a2e-40e8-a7e9-2080441441ef'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:35.228392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 932/1839] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:35 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:35.233776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7a27d5-f00e-49e7-a167-b65ffb3d0b1f None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:35.236791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-da7a27d5-f00e-49e7-a167-b65ffb3d0b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:35.252800 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-7cdf5916-8a2e-40e8-a7e9-2080441441ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca176d06824d4b76ae982cc94c1f6a4b', 'request_id': 'req-7cdf5916-8a2e-40e8-a7e9-2080441441ef'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:35.258805 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-7cdf5916-8a2e-40e8-a7e9-2080441441ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca176d06824d4b76ae982cc94c1f6a4b', 'request_id': 'req-7cdf5916-8a2e-40e8-a7e9-2080441441ef'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:35.270167 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-7cdf5916-8a2e-40e8-a7e9-2080441441ef admin admin] The token cache is being invalidate because project ca176d06824d4b76ae982cc94c1f6a4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:55:35.270449 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-7cdf5916-8a2e-40e8-a7e9-2080441441ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:35.271168 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-7cdf5916-8a2e-40e8-a7e9-2080441441ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:35.272705 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 908/1840] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:55:34 2022] DELETE /identity/v3/projects/ca176d06824d4b76ae982cc94c1f6a4b => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:35.277680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db110f8-c97a-4bbf-a5c7-fa09aea8107f None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:35.277935 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db110f8-c97a-4bbf-a5c7-fa09aea8107f None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:35.278218 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db110f8-c97a-4bbf-a5c7-fa09aea8107f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:35.299267 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-9db110f8-c97a-4bbf-a5c7-fa09aea8107f None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:35.300573 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-9db110f8-c97a-4bbf-a5c7-fa09aea8107f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:35.304258 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7a27d5-f00e-49e7-a167-b65ffb3d0b1f None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:35.376216 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 909/1841] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:55:35.384522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51baca0e-bdaa-4cec-ba95-eeb668f4e5db None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:35.384862 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51baca0e-bdaa-4cec-ba95-eeb668f4e5db None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:35.385144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51baca0e-bdaa-4cec-ba95-eeb668f4e5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:35.389099 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3338b9-607d-45e4-b45d-d804d3f5163a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:35.391234 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3338b9-607d-45e4-b45d-d804d3f5163a admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:35.391234 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3338b9-607d-45e4-b45d-d804d3f5163a admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:35.392523 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3338b9-607d-45e4-b45d-d804d3f5163a admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:35.394083 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3338b9-607d-45e4-b45d-d804d3f5163a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:35.396738 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3338b9-607d-45e4-b45d-d804d3f5163a admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:35.420374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-51baca0e-bdaa-4cec-ba95-eeb668f4e5db None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:35.422512 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-51baca0e-bdaa-4cec-ba95-eeb668f4e5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:35.441299 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 910/1842] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:55:35.446147 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51baca0e-bdaa-4cec-ba95-eeb668f4e5db None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:35.469138 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51baca0e-bdaa-4cec-ba95-eeb668f4e5db None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:35.474741 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a271c-1133-491a-b4d5-95424847af7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:35.475846 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a271c-1133-491a-b4d5-95424847af7c admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:35.475846 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a271c-1133-491a-b4d5-95424847af7c admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:35.476078 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690a271c-1133-491a-b4d5-95424847af7c admin admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:35.476615 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a271c-1133-491a-b4d5-95424847af7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:55:35.476915 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a271c-1133-491a-b4d5-95424847af7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:35.478684 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690a271c-1133-491a-b4d5-95424847af7c admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:35.498128 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 911/1843] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:55:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:55:35.499915 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a271c-1133-491a-b4d5-95424847af7c admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:35.514376 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690a271c-1133-491a-b4d5-95424847af7c admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:35.520744 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 933/1844] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:55:35 2022] POST /identity/v3/projects => generated 394 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:55:35.524985 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cce302-ce05-4385-b540-14e4b2dc6621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:35.525378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3338b9-607d-45e4-b45d-d804d3f5163a admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:35.525792 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cce302-ce05-4385-b540-14e4b2dc6621 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:35.526223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cce302-ce05-4385-b540-14e4b2dc6621 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:35.526571 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cce302-ce05-4385-b540-14e4b2dc6621 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:35.527327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cce302-ce05-4385-b540-14e4b2dc6621 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:35.529486 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-ce3338b9-607d-45e4-b45d-d804d3f5163a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:35.530423 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cce302-ce05-4385-b540-14e4b2dc6621 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:35.624760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 912/1845] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:55:35 2022] POST /identity/v3/projects => generated 358 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:35.626396 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cce302-ce05-4385-b540-14e4b2dc6621 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:35.630946 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3338b9-607d-45e4-b45d-d804d3f5163a admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:35.730516 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31176f48-04d3-4d20-b245-3de5d8192c66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:35.731518 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31176f48-04d3-4d20-b245-3de5d8192c66 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:35.731806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31176f48-04d3-4d20-b245-3de5d8192c66 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:35.732082 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31176f48-04d3-4d20-b245-3de5d8192c66 admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:35.732763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31176f48-04d3-4d20-b245-3de5d8192c66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:35.736911 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31176f48-04d3-4d20-b245-3de5d8192c66 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:35.755126 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cce302-ce05-4385-b540-14e4b2dc6621 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:35.770869 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56781dc-2a26-4743-878d-713df9b45662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:35.770869 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56781dc-2a26-4743-878d-713df9b45662 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:35.770869 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56781dc-2a26-4743-878d-713df9b45662 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:35.771515 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56781dc-2a26-4743-878d-713df9b45662 admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:35.772679 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56781dc-2a26-4743-878d-713df9b45662 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:35.775149 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56781dc-2a26-4743-878d-713df9b45662 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:35.835985 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 934/1846] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:55:35 2022] POST /identity/v3/users => generated 304 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:55:35.840612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31176f48-04d3-4d20-b245-3de5d8192c66 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:35.856234 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31176f48-04d3-4d20-b245-3de5d8192c66 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:35.862102 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da6ea3-2273-4e05-a320-15866aae8104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:35.862769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da6ea3-2273-4e05-a320-15866aae8104 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:35.863081 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da6ea3-2273-4e05-a320-15866aae8104 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:35.863371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da6ea3-2273-4e05-a320-15866aae8104 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:35.865588 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da6ea3-2273-4e05-a320-15866aae8104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:35.871709 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2da6ea3-2273-4e05-a320-15866aae8104 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:35.871808 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 913/1847] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:55:35 2022] POST /identity/v3/users => generated 280 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:55:35.875765 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56781dc-2a26-4743-878d-713df9b45662 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:35.877644 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-c56781dc-2a26-4743-878d-713df9b45662 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:35.891718 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 935/1848] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:35 2022] GET /identity/v3/roles => generated 1427 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:35.894197 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da6ea3-2273-4e05-a320-15866aae8104 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:35.906167 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56781dc-2a26-4743-878d-713df9b45662 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:35.907859 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da6ea3-2273-4e05-a320-15866aae8104 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:35.915548 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ac246-cb16-4939-a251-309901c38e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:35.919923 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f908b-1bc0-4ad4-81b3-1b6aabe1c0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:35.920837 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f908b-1bc0-4ad4-81b3-1b6aabe1c0d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:35.921788 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f908b-1bc0-4ad4-81b3-1b6aabe1c0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:35.921788 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02f908b-1bc0-4ad4-81b3-1b6aabe1c0d8 admin admin] PATH_INFO: `/v3/projects/c3552c8e478a450da459b5546f5de039` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:35.924181 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02f908b-1bc0-4ad4-81b3-1b6aabe1c0d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3552c8e478a450da459b5546f5de039)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:35.926828 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02f908b-1bc0-4ad4-81b3-1b6aabe1c0d8 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:35.929150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:35.941810 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ac246-cb16-4939-a251-309901c38e3e admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:35.942106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ac246-cb16-4939-a251-309901c38e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:35.942386 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54ac246-cb16-4939-a251-309901c38e3e admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:35.943815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ac246-cb16-4939-a251-309901c38e3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:35.945788 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54ac246-cb16-4939-a251-309901c38e3e admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:35.960096 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 936/1849] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:35 2022] GET /identity/v3/roles => generated 1427 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:35.961080 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ac246-cb16-4939-a251-309901c38e3e admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:35.973487 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54ac246-cb16-4939-a251-309901c38e3e admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:35.978885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a6804b-4f02-47fa-8ca1-5602bd871557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:35.979633 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6804b-4f02-47fa-8ca1-5602bd871557 admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:35.980117 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6804b-4f02-47fa-8ca1-5602bd871557 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:35.980567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a6804b-4f02-47fa-8ca1-5602bd871557 admin admin] PATH_INFO: `/v3/projects/8e3aa72e1e494f98b278edddee59f86d/users/31c78db97916413ab8b6b15b13ee7e1b/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:35.998483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a6804b-4f02-47fa-8ca1-5602bd871557 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e3aa72e1e494f98b278edddee59f86d, user_id=31c78db97916413ab8b6b15b13ee7e1b, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:36.005690 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a6804b-4f02-47fa-8ca1-5602bd871557 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:36.053731 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:36.061058 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 937/1850] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:35 2022] PUT /identity/v3/projects/8e3aa72e1e494f98b278edddee59f86d/users/31c78db97916413ab8b6b15b13ee7e1b/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:36.062313 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a6804b-4f02-47fa-8ca1-5602bd871557 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:36.086222 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-b02f908b-1bc0-4ad4-81b3-1b6aabe1c0d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3552c8e478a450da459b5546f5de039', 'request_id': 'req-b02f908b-1bc0-4ad4-81b3-1b6aabe1c0d8'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:36.132799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-b02f908b-1bc0-4ad4-81b3-1b6aabe1c0d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3552c8e478a450da459b5546f5de039', 'request_id': 'req-b02f908b-1bc0-4ad4-81b3-1b6aabe1c0d8'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:36.144037 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-b02f908b-1bc0-4ad4-81b3-1b6aabe1c0d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3552c8e478a450da459b5546f5de039', 'request_id': 'req-b02f908b-1bc0-4ad4-81b3-1b6aabe1c0d8'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:36.162218 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a6804b-4f02-47fa-8ca1-5602bd871557 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:36.177185 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-b02f908b-1bc0-4ad4-81b3-1b6aabe1c0d8 admin admin] The token cache is being invalidate because project c3552c8e478a450da459b5546f5de039 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:55:36.177386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-b02f908b-1bc0-4ad4-81b3-1b6aabe1c0d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:36.178077 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-b02f908b-1bc0-4ad4-81b3-1b6aabe1c0d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:36.183935 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 914/1851] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:55:35 2022] DELETE /identity/v3/projects/c3552c8e478a450da459b5546f5de039 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:55:36.185568 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f908b-1bc0-4ad4-81b3-1b6aabe1c0d8 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:36.187203 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-b02f908b-1bc0-4ad4-81b3-1b6aabe1c0d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:36.280543 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c8be68-c673-4a62-b24d-377794210788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:36.281173 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8be68-c673-4a62-b24d-377794210788 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:36.282198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8be68-c673-4a62-b24d-377794210788 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:36.282198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8be68-c673-4a62-b24d-377794210788 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:36.282414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c8be68-c673-4a62-b24d-377794210788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:36.284059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c8be68-c673-4a62-b24d-377794210788 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:36.295723 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 938/1852] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:36 2022] GET /identity/v3/roles => generated 1427 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:36.298551 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6e592-ecbf-4fb1-80e9-238229e74ac7 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:36.300486 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6e592-ecbf-4fb1-80e9-238229e74ac7 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:36.300762 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6e592-ecbf-4fb1-80e9-238229e74ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:36.320930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-d5a6e592-ecbf-4fb1-80e9-238229e74ac7 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:36.322154 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-d5a6e592-ecbf-4fb1-80e9-238229e74ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:36.329521 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02f908b-1bc0-4ad4-81b3-1b6aabe1c0d8 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:36.333034 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 939/1853] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:36.333857 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6e592-ecbf-4fb1-80e9-238229e74ac7 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:36.336840 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-d5a6e592-ecbf-4fb1-80e9-238229e74ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:36.339430 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b3e23e-814d-42ba-b530-35d91455e050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:36.340474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b3e23e-814d-42ba-b530-35d91455e050 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:36.341793 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b3e23e-814d-42ba-b530-35d91455e050 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:36.342243 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b3e23e-814d-42ba-b530-35d91455e050 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:36.345341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b3e23e-814d-42ba-b530-35d91455e050 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:36.345341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b3e23e-814d-42ba-b530-35d91455e050 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:36.359695 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 915/1854] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:36 2022] GET /identity/v3/roles => generated 1427 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:36.362606 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f85482-6870-4cd3-b6d4-023c75508cc3 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:36.362968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f85482-6870-4cd3-b6d4-023c75508cc3 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:36.363337 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f85482-6870-4cd3-b6d4-023c75508cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:36.367448 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6e592-ecbf-4fb1-80e9-238229e74ac7 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:36.373064 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de58aa59-0aeb-4d32-8613-91fa43f4a5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:36.374158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58aa59-0aeb-4d32-8613-91fa43f4a5df admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:36.374690 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58aa59-0aeb-4d32-8613-91fa43f4a5df admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:36.375155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de58aa59-0aeb-4d32-8613-91fa43f4a5df admin admin] PATH_INFO: `/v3/projects/f7e87b8d85ce4072ba8fc7caa8144af6/users/41b74e1893f84606966996e2c0ef31e7/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:36.392252 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-f0f85482-6870-4cd3-b6d4-023c75508cc3 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:36.393737 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f0f85482-6870-4cd3-b6d4-023c75508cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:36.399963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de58aa59-0aeb-4d32-8613-91fa43f4a5df admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7e87b8d85ce4072ba8fc7caa8144af6, user_id=41b74e1893f84606966996e2c0ef31e7, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:36.402745 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de58aa59-0aeb-4d32-8613-91fa43f4a5df admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:36.410870 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 916/1855] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:36.411744 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f85482-6870-4cd3-b6d4-023c75508cc3 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:36.429813 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f85482-6870-4cd3-b6d4-023c75508cc3 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:36.440511 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123bd279-726d-40ec-81d3-cb5bcf293dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:36.441198 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bd279-726d-40ec-81d3-cb5bcf293dc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:36.441475 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bd279-726d-40ec-81d3-cb5bcf293dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:36.441747 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123bd279-726d-40ec-81d3-cb5bcf293dc3 admin admin] PATH_INFO: `/v3/projects/8e3aa72e1e494f98b278edddee59f86d/users/31c78db97916413ab8b6b15b13ee7e1b/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:36.444007 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123bd279-726d-40ec-81d3-cb5bcf293dc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e3aa72e1e494f98b278edddee59f86d, user_id=31c78db97916413ab8b6b15b13ee7e1b, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:36.446757 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123bd279-726d-40ec-81d3-cb5bcf293dc3 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:36.468091 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 917/1856] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:36 2022] PUT /identity/v3/projects/8e3aa72e1e494f98b278edddee59f86d/users/31c78db97916413ab8b6b15b13ee7e1b/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:36.468970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123bd279-726d-40ec-81d3-cb5bcf293dc3 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:36.476361 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 940/1857] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:36 2022] PUT /identity/v3/projects/f7e87b8d85ce4072ba8fc7caa8144af6/users/41b74e1893f84606966996e2c0ef31e7/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:36.480530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed6aa4-5e96-494c-865b-2f1dcfeab435 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:36.480846 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed6aa4-5e96-494c-865b-2f1dcfeab435 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:36.483483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed6aa4-5e96-494c-865b-2f1dcfeab435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:36.524029 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-2aed6aa4-5e96-494c-865b-2f1dcfeab435 None None] MFA Rules not processed for user `31c78db97916413ab8b6b15b13ee7e1b`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:36.525270 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-2aed6aa4-5e96-494c-865b-2f1dcfeab435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:36.604292 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123bd279-726d-40ec-81d3-cb5bcf293dc3 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:36.657134 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 941/1858] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:36 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:36.657134 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62921f07-4b7f-4570-a53d-68b921689f3d None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:36.657134 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62921f07-4b7f-4570-a53d-68b921689f3d None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:36.657134 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62921f07-4b7f-4570-a53d-68b921689f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:36.687883 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-62921f07-4b7f-4570-a53d-68b921689f3d None None] MFA Rules not processed for user `41b74e1893f84606966996e2c0ef31e7`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:36.689015 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-62921f07-4b7f-4570-a53d-68b921689f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:36.699119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c1413-c187-49fe-97d3-048d67609489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:36.701679 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c1413-c187-49fe-97d3-048d67609489 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:36.701982 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c1413-c187-49fe-97d3-048d67609489 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:36.702270 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41c1413-c187-49fe-97d3-048d67609489 admin admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:36.702988 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c1413-c187-49fe-97d3-048d67609489 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:55:36.703329 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c1413-c187-49fe-97d3-048d67609489 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:36.704975 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41c1413-c187-49fe-97d3-048d67609489 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:36.721366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 918/1859] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:55:36 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:55:36.721366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c1413-c187-49fe-97d3-048d67609489 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:36.721366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-b41c1413-c187-49fe-97d3-048d67609489 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:36.763852 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:36.795956 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41c1413-c187-49fe-97d3-048d67609489 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:36.800110 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b58e47-06d3-4590-90b2-46aaa7655998 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:36.800809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b58e47-06d3-4590-90b2-46aaa7655998 service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:36.801939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b58e47-06d3-4590-90b2-46aaa7655998 service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:36.801939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b58e47-06d3-4590-90b2-46aaa7655998 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:36.812008 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b58e47-06d3-4590-90b2-46aaa7655998 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:36.813510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b58e47-06d3-4590-90b2-46aaa7655998 service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:36.826769 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 919/1860] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:36 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:36.828115 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b58e47-06d3-4590-90b2-46aaa7655998 service cinder] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:36.834986 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 942/1861] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:36.843654 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62921f07-4b7f-4570-a53d-68b921689f3d None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:36.845297 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b58e47-06d3-4590-90b2-46aaa7655998 service cinder] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:36.858605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1ddd1-a1eb-4ea4-8b44-4dc02944a1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:36.859271 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1ddd1-a1eb-4ea4-8b44-4dc02944a1ae admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:36.859738 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1ddd1-a1eb-4ea4-8b44-4dc02944a1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:36.864283 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1ddd1-a1eb-4ea4-8b44-4dc02944a1ae admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:36.864283 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a1ddd1-a1eb-4ea4-8b44-4dc02944a1ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:36.864283 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a1ddd1-a1eb-4ea4-8b44-4dc02944a1ae admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:36.901662 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62921f07-4b7f-4570-a53d-68b921689f3d None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:36.912006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317a913-029f-4fd7-8d5a-b74673276bd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:36.913856 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317a913-029f-4fd7-8d5a-b74673276bd1 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:36.915303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317a913-029f-4fd7-8d5a-b74673276bd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:36.916038 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317a913-029f-4fd7-8d5a-b74673276bd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:36.918320 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317a913-029f-4fd7-8d5a-b74673276bd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:36.920801 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317a913-029f-4fd7-8d5a-b74673276bd1 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:36.937658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:36.990048 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 920/1862] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:55:36 2022] POST /identity/v3/projects => generated 388 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:36.990048 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1ddd1-a1eb-4ea4-8b44-4dc02944a1ae admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:37.033172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 943/1863] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:55:36 2022] POST /identity/v3/projects => generated 356 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:55:37.034266 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317a913-029f-4fd7-8d5a-b74673276bd1 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:37.058944 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1ddd1-a1eb-4ea4-8b44-4dc02944a1ae admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:37.119443 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317a913-029f-4fd7-8d5a-b74673276bd1 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:37.124909 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ce6e4-4d50-4022-840f-2a79bbf9ae76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:37.125337 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529f91a-a31c-45a6-b411-ff11cb8f974b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:37.125593 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ce6e4-4d50-4022-840f-2a79bbf9ae76 service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:37.125872 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ce6e4-4d50-4022-840f-2a79bbf9ae76 service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:37.125989 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529f91a-a31c-45a6-b411-ff11cb8f974b admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:37.126220 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529f91a-a31c-45a6-b411-ff11cb8f974b admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:37.126319 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9ce6e4-4d50-4022-840f-2a79bbf9ae76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:37.126825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529f91a-a31c-45a6-b411-ff11cb8f974b admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:37.127217 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d529f91a-a31c-45a6-b411-ff11cb8f974b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:37.128320 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-eb9ce6e4-4d50-4022-840f-2a79bbf9ae76 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:37.129165 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d529f91a-a31c-45a6-b411-ff11cb8f974b admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:37.213088 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 944/1864] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:55:37 2022] POST /identity/v3/users => generated 295 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:37.214253 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529f91a-a31c-45a6-b411-ff11cb8f974b admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:37.217287 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9ce6e4-4d50-4022-840f-2a79bbf9ae76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:37.219115 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9ce6e4-4d50-4022-840f-2a79bbf9ae76 service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:37.229282 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529f91a-a31c-45a6-b411-ff11cb8f974b admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:37.231555 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 921/1865] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:36 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:37.232618 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ce6e4-4d50-4022-840f-2a79bbf9ae76 service cinder] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:37.234996 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318cc3f-aacc-465a-a5d3-bff6e0852ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:37.235772 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318cc3f-aacc-465a-a5d3-bff6e0852ec3 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:37.236124 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318cc3f-aacc-465a-a5d3-bff6e0852ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:37.236516 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3318cc3f-aacc-465a-a5d3-bff6e0852ec3 admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:37.237251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3318cc3f-aacc-465a-a5d3-bff6e0852ec3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:37.239520 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3318cc3f-aacc-465a-a5d3-bff6e0852ec3 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:37.251640 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9ce6e4-4d50-4022-840f-2a79bbf9ae76 service cinder] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:37.279903 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14645e16-2c23-4c3f-95ba-2eb4a3348235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:37.280547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14645e16-2c23-4c3f-95ba-2eb4a3348235 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:37.280829 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14645e16-2c23-4c3f-95ba-2eb4a3348235 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:37.281103 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14645e16-2c23-4c3f-95ba-2eb4a3348235 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:37.281880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14645e16-2c23-4c3f-95ba-2eb4a3348235 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:37.283704 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14645e16-2c23-4c3f-95ba-2eb4a3348235 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:37.307485 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 922/1866] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:37 2022] GET /identity/v3/roles => generated 1427 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:37.308573 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14645e16-2c23-4c3f-95ba-2eb4a3348235 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:37.333123 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14645e16-2c23-4c3f-95ba-2eb4a3348235 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:37.340374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab76f6-41dc-4be8-a619-f4d92fa27ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:37.341215 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab76f6-41dc-4be8-a619-f4d92fa27ccc admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:37.341505 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab76f6-41dc-4be8-a619-f4d92fa27ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:37.342165 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab76f6-41dc-4be8-a619-f4d92fa27ccc admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:37.343393 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab76f6-41dc-4be8-a619-f4d92fa27ccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:37.358481 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab76f6-41dc-4be8-a619-f4d92fa27ccc admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:37.380959 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 945/1867] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:55:37 2022] POST /identity/v3/users => generated 285 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:37.385047 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318cc3f-aacc-465a-a5d3-bff6e0852ec3 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:37.395454 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 923/1868] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:37 2022] GET /identity/v3/roles => generated 1427 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:37.396427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab76f6-41dc-4be8-a619-f4d92fa27ccc admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:37.404628 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3318cc3f-aacc-465a-a5d3-bff6e0852ec3 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:37.413865 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9b9fe-0a5c-496b-9dda-bdbe61cd66fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:37.414659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9b9fe-0a5c-496b-9dda-bdbe61cd66fa admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:37.415013 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9b9fe-0a5c-496b-9dda-bdbe61cd66fa admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:37.415359 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab76f6-41dc-4be8-a619-f4d92fa27ccc admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:37.415479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a9b9fe-0a5c-496b-9dda-bdbe61cd66fa admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:37.416286 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9b9fe-0a5c-496b-9dda-bdbe61cd66fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:37.420198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a9b9fe-0a5c-496b-9dda-bdbe61cd66fa admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:37.429912 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5c9c66-86a0-4236-aa5f-ce9c824386bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:37.430613 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c9c66-86a0-4236-aa5f-ce9c824386bc admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:37.438384 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c9c66-86a0-4236-aa5f-ce9c824386bc admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:37.438565 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5c9c66-86a0-4236-aa5f-ce9c824386bc admin admin] PATH_INFO: `/v3/projects/e2b438cd0b9d44d69e27696ce0dc6ca3/users/ec283070309443afbbb4da5c452d2dc5/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:37.452201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 946/1869] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:37 2022] GET /identity/v3/roles => generated 1427 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:55:37.453159 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9b9fe-0a5c-496b-9dda-bdbe61cd66fa admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:37.478259 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5c9c66-86a0-4236-aa5f-ce9c824386bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2b438cd0b9d44d69e27696ce0dc6ca3, user_id=ec283070309443afbbb4da5c452d2dc5, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:37.478534 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a9b9fe-0a5c-496b-9dda-bdbe61cd66fa admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:37.481683 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5c9c66-86a0-4236-aa5f-ce9c824386bc admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:37.486136 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8677a-a74d-4fd8-9466-104d2cfd42be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:37.489017 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8677a-a74d-4fd8-9466-104d2cfd42be admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:37.489017 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8677a-a74d-4fd8-9466-104d2cfd42be admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:37.489017 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8677a-a74d-4fd8-9466-104d2cfd42be admin admin] PATH_INFO: `/v3/projects/525584e541654e8788d53de31d384d34/users/13efd1206d3c42e4ae773b6ae8a306db/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:37.505479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8677a-a74d-4fd8-9466-104d2cfd42be admin admin] RBAC: Authorizing `identity:create_grant(project_id=525584e541654e8788d53de31d384d34, user_id=13efd1206d3c42e4ae773b6ae8a306db, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:37.513512 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8677a-a74d-4fd8-9466-104d2cfd42be admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:37.563431 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 924/1870] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:37 2022] PUT /identity/v3/projects/e2b438cd0b9d44d69e27696ce0dc6ca3/users/ec283070309443afbbb4da5c452d2dc5/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:37.574295 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339250b-b311-4d0a-a02c-139917925dfa None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:37.574585 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339250b-b311-4d0a-a02c-139917925dfa None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:37.574864 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8339250b-b311-4d0a-a02c-139917925dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:37.585056 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 947/1871] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:37 2022] PUT /identity/v3/projects/525584e541654e8788d53de31d384d34/users/13efd1206d3c42e4ae773b6ae8a306db/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:37.592136 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8677a-a74d-4fd8-9466-104d2cfd42be admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:37.608611 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-8339250b-b311-4d0a-a02c-139917925dfa None None] MFA Rules not processed for user `ec283070309443afbbb4da5c452d2dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:37.610030 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-8339250b-b311-4d0a-a02c-139917925dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:37.662122 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8677a-a74d-4fd8-9466-104d2cfd42be admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:37.667248 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf0fd5-2e61-4d74-9c68-3cd92465328b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:37.667940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf0fd5-2e61-4d74-9c68-3cd92465328b admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:37.668219 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf0fd5-2e61-4d74-9c68-3cd92465328b admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:37.668462 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecf0fd5-2e61-4d74-9c68-3cd92465328b admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:37.669682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf0fd5-2e61-4d74-9c68-3cd92465328b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:37.671494 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecf0fd5-2e61-4d74-9c68-3cd92465328b admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:37.691238 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 925/1872] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:37 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:37.691477 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 948/1873] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:37 2022] GET /identity/v3/roles => generated 1427 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:37.695378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf0fd5-2e61-4d74-9c68-3cd92465328b admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:37.696501 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339250b-b311-4d0a-a02c-139917925dfa None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:37.706819 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecf0fd5-2e61-4d74-9c68-3cd92465328b admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:37.711626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af4796-ff76-4ca7-84d5-4ec08ff10915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:37.712272 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af4796-ff76-4ca7-84d5-4ec08ff10915 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:37.712553 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af4796-ff76-4ca7-84d5-4ec08ff10915 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:37.712829 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af4796-ff76-4ca7-84d5-4ec08ff10915 admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:37.712926 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8339250b-b311-4d0a-a02c-139917925dfa None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:37.713540 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af4796-ff76-4ca7-84d5-4ec08ff10915 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:37.715489 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af4796-ff76-4ca7-84d5-4ec08ff10915 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:37.719491 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516bf9fc-0d3d-4a5c-8cf9-a12a61bace5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:37.720370 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516bf9fc-0d3d-4a5c-8cf9-a12a61bace5a admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:37.720670 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516bf9fc-0d3d-4a5c-8cf9-a12a61bace5a admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:37.720908 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516bf9fc-0d3d-4a5c-8cf9-a12a61bace5a admin admin] PATH_INFO: `/v3/projects/525584e541654e8788d53de31d384d34/users/13efd1206d3c42e4ae773b6ae8a306db/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:37.722997 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516bf9fc-0d3d-4a5c-8cf9-a12a61bace5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=525584e541654e8788d53de31d384d34, user_id=13efd1206d3c42e4ae773b6ae8a306db, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:37.726735 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516bf9fc-0d3d-4a5c-8cf9-a12a61bace5a admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:37.740219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 926/1874] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:37 2022] PUT /identity/v3/projects/525584e541654e8788d53de31d384d34/users/13efd1206d3c42e4ae773b6ae8a306db/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:55:37.750984 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a175b-033b-4b9c-adb1-5644c2a28987 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:37.751304 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a175b-033b-4b9c-adb1-5644c2a28987 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:37.751579 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9a175b-033b-4b9c-adb1-5644c2a28987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:37.780721 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-cd9a175b-033b-4b9c-adb1-5644c2a28987 None None] MFA Rules not processed for user `13efd1206d3c42e4ae773b6ae8a306db`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:37.781380 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-cd9a175b-033b-4b9c-adb1-5644c2a28987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:37.853930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 949/1875] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:55:37 2022] POST /identity/v3/projects => generated 388 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:37.854346 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 927/1876] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:37 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:55:37.858846 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af4796-ff76-4ca7-84d5-4ec08ff10915 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:37.872660 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a175b-033b-4b9c-adb1-5644c2a28987 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:37.874126 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-cd9a175b-033b-4b9c-adb1-5644c2a28987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:37.952173 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9a175b-033b-4b9c-adb1-5644c2a28987 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:37.989982 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af4796-ff76-4ca7-84d5-4ec08ff10915 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:38.013125 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd66615-fc16-46c9-9edf-3911a8a690bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:38.013457 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd66615-fc16-46c9-9edf-3911a8a690bd admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:38.013917 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd66615-fc16-46c9-9edf-3911a8a690bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:38.018367 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd66615-fc16-46c9-9edf-3911a8a690bd admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:38.018367 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd66615-fc16-46c9-9edf-3911a8a690bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:38.018367 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd66615-fc16-46c9-9edf-3911a8a690bd admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:38.058191 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399c842-6c53-454a-a09a-87141a6ac291 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:38.066470 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399c842-6c53-454a-a09a-87141a6ac291 service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:38.067756 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399c842-6c53-454a-a09a-87141a6ac291 service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:38.068377 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1399c842-6c53-454a-a09a-87141a6ac291 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:38.153783 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 950/1877] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:55:37 2022] POST /identity/v3/users => generated 301 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:38.157736 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd66615-fc16-46c9-9edf-3911a8a690bd admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:38.173338 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd66615-fc16-46c9-9edf-3911a8a690bd admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:38.176656 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1399c842-6c53-454a-a09a-87141a6ac291 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:38.178873 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1399c842-6c53-454a-a09a-87141a6ac291 service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:38.186860 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd167b-0569-42eb-a73f-59ba830aeabc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:38.187673 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd167b-0569-42eb-a73f-59ba830aeabc admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:38.188027 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd167b-0569-42eb-a73f-59ba830aeabc admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:38.188237 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbd167b-0569-42eb-a73f-59ba830aeabc admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:38.188945 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbd167b-0569-42eb-a73f-59ba830aeabc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:38.190635 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbd167b-0569-42eb-a73f-59ba830aeabc admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:38.194816 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 928/1878] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:37 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:38.222902 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 951/1879] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:38 2022] GET /identity/v3/roles => generated 1427 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:55:38.242802 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399c842-6c53-454a-a09a-87141a6ac291 service cinder] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:38.265760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1399c842-6c53-454a-a09a-87141a6ac291 service cinder] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:38.271875 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cbc8f-de0d-4e86-bda9-28bbddec5aaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:38.272558 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cbc8f-de0d-4e86-bda9-28bbddec5aaa admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:38.272850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cbc8f-de0d-4e86-bda9-28bbddec5aaa admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:38.273194 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25cbc8f-de0d-4e86-bda9-28bbddec5aaa admin admin] PATH_INFO: `/v3/projects/0bc3cbfc64424e10a0b9428b8dfca08c/users/aadc4a838a73468484e785e1b0e955c1/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:38.291733 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25cbc8f-de0d-4e86-bda9-28bbddec5aaa admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bc3cbfc64424e10a0b9428b8dfca08c, user_id=aadc4a838a73468484e785e1b0e955c1, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:38.295391 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25cbc8f-de0d-4e86-bda9-28bbddec5aaa admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:38.407684 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:39.411201 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 929/1880] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:38 2022] PUT /identity/v3/projects/0bc3cbfc64424e10a0b9428b8dfca08c/users/aadc4a838a73468484e785e1b0e955c1/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 1168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:39.414365 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd167b-0569-42eb-a73f-59ba830aeabc admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:39.426001 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbd167b-0569-42eb-a73f-59ba830aeabc admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:39.510437 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1763413-7a90-4278-8fa1-4d69fbcb3580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:39.511206 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1763413-7a90-4278-8fa1-4d69fbcb3580 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:39.511610 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1763413-7a90-4278-8fa1-4d69fbcb3580 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:39.511957 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1763413-7a90-4278-8fa1-4d69fbcb3580 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:39.512778 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1763413-7a90-4278-8fa1-4d69fbcb3580 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:39.515314 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1763413-7a90-4278-8fa1-4d69fbcb3580 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:39.534942 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 952/1881] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:39 2022] GET /identity/v3/roles => generated 1427 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:39.539126 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cbc8f-de0d-4e86-bda9-28bbddec5aaa admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:39.542522 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1763413-7a90-4278-8fa1-4d69fbcb3580 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:39.553267 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25cbc8f-de0d-4e86-bda9-28bbddec5aaa admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:39.560816 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df8dd5d-2ec0-4719-a6d8-25cf46bf9ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:39.561752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8dd5d-2ec0-4719-a6d8-25cf46bf9ac5 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:39.562196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8dd5d-2ec0-4719-a6d8-25cf46bf9ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:39.562613 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df8dd5d-2ec0-4719-a6d8-25cf46bf9ac5 admin admin] PATH_INFO: `/v3/projects/0bc3cbfc64424e10a0b9428b8dfca08c/users/aadc4a838a73468484e785e1b0e955c1/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:39.565081 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df8dd5d-2ec0-4719-a6d8-25cf46bf9ac5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bc3cbfc64424e10a0b9428b8dfca08c, user_id=aadc4a838a73468484e785e1b0e955c1, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:39.571695 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df8dd5d-2ec0-4719-a6d8-25cf46bf9ac5 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:39.595886 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 930/1882] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:39 2022] PUT /identity/v3/projects/0bc3cbfc64424e10a0b9428b8dfca08c/users/aadc4a838a73468484e785e1b0e955c1/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:39.601693 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3e395-7be0-4db2-81b5-2b873a7ded45 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:39.601982 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3e395-7be0-4db2-81b5-2b873a7ded45 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:39.602878 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb3e395-7be0-4db2-81b5-2b873a7ded45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:39.636479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1763413-7a90-4278-8fa1-4d69fbcb3580 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:39.653918 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-1fb3e395-7be0-4db2-81b5-2b873a7ded45 None None] MFA Rules not processed for user `aadc4a838a73468484e785e1b0e955c1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:39.655343 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-1fb3e395-7be0-4db2-81b5-2b873a7ded45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:39.719109 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d26dc-7b0c-4172-80f0-014d1e57a821 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:39.719904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d26dc-7b0c-4172-80f0-014d1e57a821 service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:39.720332 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d26dc-7b0c-4172-80f0-014d1e57a821 service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:39.720673 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24d26dc-7b0c-4172-80f0-014d1e57a821 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:39.745189 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 931/1883] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:39 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:39.755835 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59fb6c-6eba-45a8-9542-4e2014422e8a None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:39.756271 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59fb6c-6eba-45a8-9542-4e2014422e8a None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:39.756558 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b59fb6c-6eba-45a8-9542-4e2014422e8a None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:39.758291 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 932/1884] 10.222.0.115 () {54 vars in 926 bytes} [Tue Aug 2 10:55:39 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (2 switches on core 0) Aug 02 10:55:39.767292 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f6e0d-20c3-4751-ba07-cc64c0013676 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:39.768417 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f6e0d-20c3-4751-ba07-cc64c0013676 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:39.768417 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10f6e0d-20c3-4751-ba07-cc64c0013676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:39.792773 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-b10f6e0d-20c3-4751-ba07-cc64c0013676 None None] MFA Rules not processed for user `d0478bd540f446ec810e2091c46ac49c`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:39.795494 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-b10f6e0d-20c3-4751-ba07-cc64c0013676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:39.838170 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:39.845190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d26dc-7b0c-4172-80f0-014d1e57a821 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:39.846957 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24d26dc-7b0c-4172-80f0-014d1e57a821 service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:39.877148 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 953/1885] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:55:39.877148 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d26dc-7b0c-4172-80f0-014d1e57a821 service cinder] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:39.896838 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24d26dc-7b0c-4172-80f0-014d1e57a821 service cinder] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:39.904079 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e5582-2314-4b52-a0af-9bfc7bdb5c8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:39.904925 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e5582-2314-4b52-a0af-9bfc7bdb5c8c service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:39.905313 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e5582-2314-4b52-a0af-9bfc7bdb5c8c service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:39.905723 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e5582-2314-4b52-a0af-9bfc7bdb5c8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:39.921190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56e5582-2314-4b52-a0af-9bfc7bdb5c8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:39.928869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56e5582-2314-4b52-a0af-9bfc7bdb5c8c service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:39.929321 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 933/1886] 10.222.0.115 () {58 vars in 1023 bytes} [Tue Aug 2 10:55:39 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:39.936111 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 954/1887] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:39 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:40.008388 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f6e0d-20c3-4751-ba07-cc64c0013676 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:40.009931 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-b10f6e0d-20c3-4751-ba07-cc64c0013676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:40.029543 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10f6e0d-20c3-4751-ba07-cc64c0013676 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:40.039737 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fe7b9c-3370-4db3-9e0a-5db61c10b7d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:40.044504 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe7b9c-3370-4db3-9e0a-5db61c10b7d7 service nova] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:40.044504 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe7b9c-3370-4db3-9e0a-5db61c10b7d7 service nova] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:40.044504 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fe7b9c-3370-4db3-9e0a-5db61c10b7d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:40.061235 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e5582-2314-4b52-a0af-9bfc7bdb5c8c service cinder] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:40.063722 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe7b9c-3370-4db3-9e0a-5db61c10b7d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:40.065838 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fe7b9c-3370-4db3-9e0a-5db61c10b7d7 service nova] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:40.081253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 934/1888] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:40 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:40.146201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e5582-2314-4b52-a0af-9bfc7bdb5c8c service cinder] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:40.151576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b19294-cc5d-4976-b374-94e44d7ac6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:40.152317 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b19294-cc5d-4976-b374-94e44d7ac6ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:40.152668 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b19294-cc5d-4976-b374-94e44d7ac6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:40.153051 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b19294-cc5d-4976-b374-94e44d7ac6ae admin admin] PATH_INFO: `/v3/users/ec283070309443afbbb4da5c452d2dc5` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:40.155333 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b19294-cc5d-4976-b374-94e44d7ac6ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec283070309443afbbb4da5c452d2dc5)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:40.157479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b19294-cc5d-4976-b374-94e44d7ac6ae admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:40.448650 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-d0b19294-cc5d-4976-b374-94e44d7ac6ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec283070309443afbbb4da5c452d2dc5'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:40.449669 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-d0b19294-cc5d-4976-b374-94e44d7ac6ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec283070309443afbbb4da5c452d2dc5'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:40.534166 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-d0b19294-cc5d-4976-b374-94e44d7ac6ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec283070309443afbbb4da5c452d2dc5'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:40.562144 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-d0b19294-cc5d-4976-b374-94e44d7ac6ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec283070309443afbbb4da5c452d2dc5'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:40.586032 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-d0b19294-cc5d-4976-b374-94e44d7ac6ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec283070309443afbbb4da5c452d2dc5'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:40.594511 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 955/1889] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:55:40 2022] DELETE /identity/v3/users/ec283070309443afbbb4da5c452d2dc5 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:55:40.598936 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a6364-8e3e-442b-9ab7-7683130be0b2 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:40.600117 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a6364-8e3e-442b-9ab7-7683130be0b2 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:40.600598 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a6364-8e3e-442b-9ab7-7683130be0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:40.629342 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-0a0a6364-8e3e-442b-9ab7-7683130be0b2 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:40.630810 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-0a0a6364-8e3e-442b-9ab7-7683130be0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:40.698395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 935/1890] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:40 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:55:40.708934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b19294-cc5d-4976-b374-94e44d7ac6ae admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:40.710478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-d0b19294-cc5d-4976-b374-94e44d7ac6ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:40.771689 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b19294-cc5d-4976-b374-94e44d7ac6ae admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:40.775588 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e962a8-0b68-4459-b3f2-baf7f7795eff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:40.780253 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e962a8-0b68-4459-b3f2-baf7f7795eff service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:40.780253 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e962a8-0b68-4459-b3f2-baf7f7795eff service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:40.780253 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e962a8-0b68-4459-b3f2-baf7f7795eff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:40.788866 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e962a8-0b68-4459-b3f2-baf7f7795eff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:40.790316 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e962a8-0b68-4459-b3f2-baf7f7795eff service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:40.812627 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 956/1891] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:40 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:41.425405 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a6364-8e3e-442b-9ab7-7683130be0b2 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:41.426915 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-0a0a6364-8e3e-442b-9ab7-7683130be0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:41.442880 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e962a8-0b68-4459-b3f2-baf7f7795eff service neutron] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:41.444364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0a6364-8e3e-442b-9ab7-7683130be0b2 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:41.445150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-16e962a8-0b68-4459-b3f2-baf7f7795eff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:41.451596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d734b848-5c99-4d3c-ae9c-8fa786966670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:41.470376 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734b848-5c99-4d3c-ae9c-8fa786966670 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:41.470376 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734b848-5c99-4d3c-ae9c-8fa786966670 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:41.470376 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d734b848-5c99-4d3c-ae9c-8fa786966670 admin admin] PATH_INFO: `/v3/users/0f59b356948c4ace8f26a6436051f50e` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:41.470376 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734b848-5c99-4d3c-ae9c-8fa786966670 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f59b356948c4ace8f26a6436051f50e)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:41.470376 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d734b848-5c99-4d3c-ae9c-8fa786966670 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:41.477041 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e962a8-0b68-4459-b3f2-baf7f7795eff service neutron] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:41.485954 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2b80c-f438-4edf-b4b5-effc5ed25a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:41.486854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2b80c-f438-4edf-b4b5-effc5ed25a86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:41.487489 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2b80c-f438-4edf-b4b5-effc5ed25a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:41.488813 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2b80c-f438-4edf-b4b5-effc5ed25a86 admin admin] PATH_INFO: `/v3/projects/e2b438cd0b9d44d69e27696ce0dc6ca3` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:41.490530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2b80c-f438-4edf-b4b5-effc5ed25a86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2b438cd0b9d44d69e27696ce0dc6ca3)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:41.493117 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b2b80c-f438-4edf-b4b5-effc5ed25a86 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:41.704153 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-d734b848-5c99-4d3c-ae9c-8fa786966670 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f59b356948c4ace8f26a6436051f50e'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:41.710595 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c2b2b80c-f438-4edf-b4b5-effc5ed25a86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2b438cd0b9d44d69e27696ce0dc6ca3', 'request_id': 'req-c2b2b80c-f438-4edf-b4b5-effc5ed25a86'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:41.712931 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-d734b848-5c99-4d3c-ae9c-8fa786966670 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f59b356948c4ace8f26a6436051f50e'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:41.713829 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-d734b848-5c99-4d3c-ae9c-8fa786966670 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f59b356948c4ace8f26a6436051f50e'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:41.720260 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c2b2b80c-f438-4edf-b4b5-effc5ed25a86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2b438cd0b9d44d69e27696ce0dc6ca3', 'request_id': 'req-c2b2b80c-f438-4edf-b4b5-effc5ed25a86'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:41.749813 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:41.778311 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-d734b848-5c99-4d3c-ae9c-8fa786966670 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f59b356948c4ace8f26a6436051f50e'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:41.805242 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-d734b848-5c99-4d3c-ae9c-8fa786966670 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f59b356948c4ace8f26a6436051f50e'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:41.820444 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 936/1892] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:55:41 2022] DELETE /identity/v3/users/0f59b356948c4ace8f26a6436051f50e => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:41.827385 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3a28f-53ad-4bb4-b0f1-fdb92ff50c40 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:41.827672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3a28f-53ad-4bb4-b0f1-fdb92ff50c40 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:41.827973 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b3a28f-53ad-4bb4-b0f1-fdb92ff50c40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:41.881222 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c2b2b80c-f438-4edf-b4b5-effc5ed25a86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2b438cd0b9d44d69e27696ce0dc6ca3', 'request_id': 'req-c2b2b80c-f438-4edf-b4b5-effc5ed25a86'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:41.885815 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:41.892420 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-c4b3a28f-53ad-4bb4-b0f1-fdb92ff50c40 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:41.893674 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-c4b3a28f-53ad-4bb4-b0f1-fdb92ff50c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:41.903191 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c2b2b80c-f438-4edf-b4b5-effc5ed25a86 admin admin] The token cache is being invalidate because project e2b438cd0b9d44d69e27696ce0dc6ca3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:55:41.903521 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c2b2b80c-f438-4edf-b4b5-effc5ed25a86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:41.905270 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c2b2b80c-f438-4edf-b4b5-effc5ed25a86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:41.912947 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 957/1893] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:55:41 2022] DELETE /identity/v3/projects/e2b438cd0b9d44d69e27696ce0dc6ca3 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:41.917322 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2b80c-f438-4edf-b4b5-effc5ed25a86 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:41.920453 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-c2b2b80c-f438-4edf-b4b5-effc5ed25a86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:42.033102 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2b80c-f438-4edf-b4b5-effc5ed25a86 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:42.037675 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 937/1894] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:41 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:42.042267 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400d755-254e-4feb-8179-ee6874b2b96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:42.042967 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400d755-254e-4feb-8179-ee6874b2b96e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:42.046319 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400d755-254e-4feb-8179-ee6874b2b96e admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:42.046319 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5400d755-254e-4feb-8179-ee6874b2b96e admin admin] PATH_INFO: `/v3/users/aadc4a838a73468484e785e1b0e955c1` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:42.046319 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5400d755-254e-4feb-8179-ee6874b2b96e admin admin] RBAC: Authorizing `identity:delete_user(user_id=aadc4a838a73468484e785e1b0e955c1)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:42.052303 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3a28f-53ad-4bb4-b0f1-fdb92ff50c40 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:42.054048 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-c4b3a28f-53ad-4bb4-b0f1-fdb92ff50c40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:42.057745 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5400d755-254e-4feb-8179-ee6874b2b96e admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:42.131322 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b3a28f-53ad-4bb4-b0f1-fdb92ff50c40 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:42.141142 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcddbbc0-c4fa-4991-8fe9-eede322537bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:42.143068 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcddbbc0-c4fa-4991-8fe9-eede322537bc service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:42.143068 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcddbbc0-c4fa-4991-8fe9-eede322537bc service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:42.143988 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcddbbc0-c4fa-4991-8fe9-eede322537bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:42.153955 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcddbbc0-c4fa-4991-8fe9-eede322537bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:42.160270 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcddbbc0-c4fa-4991-8fe9-eede322537bc service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:42.166446 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 938/1895] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:42 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:55:42.288536 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-5400d755-254e-4feb-8179-ee6874b2b96e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aadc4a838a73468484e785e1b0e955c1'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:42.290415 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-5400d755-254e-4feb-8179-ee6874b2b96e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aadc4a838a73468484e785e1b0e955c1'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:42.391649 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-5400d755-254e-4feb-8179-ee6874b2b96e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aadc4a838a73468484e785e1b0e955c1'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:42.406890 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-5400d755-254e-4feb-8179-ee6874b2b96e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aadc4a838a73468484e785e1b0e955c1'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:42.430850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-5400d755-254e-4feb-8179-ee6874b2b96e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aadc4a838a73468484e785e1b0e955c1'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:42.442004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 958/1896] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:55:41 2022] DELETE /identity/v3/users/aadc4a838a73468484e785e1b0e955c1 => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:42.880182 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcddbbc0-c4fa-4991-8fe9-eede322537bc service neutron] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:42.881684 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-fcddbbc0-c4fa-4991-8fe9-eede322537bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:42.909039 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcddbbc0-c4fa-4991-8fe9-eede322537bc service neutron] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:42.971939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ae530-04e2-4346-8225-cb1f118f768f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:42.972152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ae530-04e2-4346-8225-cb1f118f768f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:42.972464 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ae530-04e2-4346-8225-cb1f118f768f admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:42.972789 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ae530-04e2-4346-8225-cb1f118f768f admin admin] PATH_INFO: `/v3/projects/3d32e15bfa92410bb1d89155c3b7aa85` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:42.974437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3ae530-04e2-4346-8225-cb1f118f768f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d32e15bfa92410bb1d89155c3b7aa85)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:42.976430 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3ae530-04e2-4346-8225-cb1f118f768f admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:43.003883 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400d755-254e-4feb-8179-ee6874b2b96e admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:43.005509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-5400d755-254e-4feb-8179-ee6874b2b96e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:43.020631 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5400d755-254e-4feb-8179-ee6874b2b96e admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:43.025723 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23c8243-3b14-4384-87db-e71f1b1437f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:43.026380 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c8243-3b14-4384-87db-e71f1b1437f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:43.026666 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c8243-3b14-4384-87db-e71f1b1437f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:43.026945 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23c8243-3b14-4384-87db-e71f1b1437f3 admin admin] PATH_INFO: `/v3/projects/0bc3cbfc64424e10a0b9428b8dfca08c` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:43.028300 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23c8243-3b14-4384-87db-e71f1b1437f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bc3cbfc64424e10a0b9428b8dfca08c)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:43.030250 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23c8243-3b14-4384-87db-e71f1b1437f3 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:43.059089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:43.114209 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-5c3ae530-04e2-4346-8225-cb1f118f768f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d32e15bfa92410bb1d89155c3b7aa85', 'request_id': 'req-5c3ae530-04e2-4346-8225-cb1f118f768f'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:43.202126 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-5c3ae530-04e2-4346-8225-cb1f118f768f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d32e15bfa92410bb1d89155c3b7aa85', 'request_id': 'req-5c3ae530-04e2-4346-8225-cb1f118f768f'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:43.208547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-5c3ae530-04e2-4346-8225-cb1f118f768f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d32e15bfa92410bb1d89155c3b7aa85', 'request_id': 'req-5c3ae530-04e2-4346-8225-cb1f118f768f'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:43.226828 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-5c3ae530-04e2-4346-8225-cb1f118f768f admin admin] The token cache is being invalidate because project 3d32e15bfa92410bb1d89155c3b7aa85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:55:43.227155 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-5c3ae530-04e2-4346-8225-cb1f118f768f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:43.227969 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-5c3ae530-04e2-4346-8225-cb1f118f768f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:43.229468 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c23c8243-3b14-4384-87db-e71f1b1437f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bc3cbfc64424e10a0b9428b8dfca08c', 'request_id': 'req-c23c8243-3b14-4384-87db-e71f1b1437f3'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:43.231472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 939/1897] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:55:42 2022] DELETE /identity/v3/projects/3d32e15bfa92410bb1d89155c3b7aa85 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:43.234091 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ae530-04e2-4346-8225-cb1f118f768f admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:43.236446 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-5c3ae530-04e2-4346-8225-cb1f118f768f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:43.237852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c23c8243-3b14-4384-87db-e71f1b1437f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bc3cbfc64424e10a0b9428b8dfca08c', 'request_id': 'req-c23c8243-3b14-4384-87db-e71f1b1437f3'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:43.264364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:43.341201 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ae530-04e2-4346-8225-cb1f118f768f admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:43.348830 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c23c8243-3b14-4384-87db-e71f1b1437f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bc3cbfc64424e10a0b9428b8dfca08c', 'request_id': 'req-c23c8243-3b14-4384-87db-e71f1b1437f3'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:43.361756 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c23c8243-3b14-4384-87db-e71f1b1437f3 admin admin] The token cache is being invalidate because project 0bc3cbfc64424e10a0b9428b8dfca08c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:55:43.362069 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c23c8243-3b14-4384-87db-e71f1b1437f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:43.363135 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c23c8243-3b14-4384-87db-e71f1b1437f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:43.368092 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 959/1898] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:55:43 2022] DELETE /identity/v3/projects/0bc3cbfc64424e10a0b9428b8dfca08c => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:55:43.369751 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d8ada-ac77-427a-8fa7-a2502144c2a2 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:43.370037 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d8ada-ac77-427a-8fa7-a2502144c2a2 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:43.370359 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266d8ada-ac77-427a-8fa7-a2502144c2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:43.394980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-266d8ada-ac77-427a-8fa7-a2502144c2a2 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:43.396909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-266d8ada-ac77-427a-8fa7-a2502144c2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:43.430932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753029a6-8143-47e1-8787-e5d0b5ce17ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:43.438582 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753029a6-8143-47e1-8787-e5d0b5ce17ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:43.441174 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753029a6-8143-47e1-8787-e5d0b5ce17ff admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:43.441174 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753029a6-8143-47e1-8787-e5d0b5ce17ff admin admin] PATH_INFO: `/v3/users/56d64a36a58e4ebb88e7f38d448b8da9` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:43.441366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753029a6-8143-47e1-8787-e5d0b5ce17ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=56d64a36a58e4ebb88e7f38d448b8da9)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:43.443698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753029a6-8143-47e1-8787-e5d0b5ce17ff admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:43.500512 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 960/1899] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:43 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:43.508113 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc716f-7b11-48d1-8e1e-983f6116a7eb None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:43.508401 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc716f-7b11-48d1-8e1e-983f6116a7eb None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:43.508736 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc716f-7b11-48d1-8e1e-983f6116a7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:43.532686 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-1ebc716f-7b11-48d1-8e1e-983f6116a7eb None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:43.533981 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-1ebc716f-7b11-48d1-8e1e-983f6116a7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:43.544402 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 961/1900] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:43 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:43.550334 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc716f-7b11-48d1-8e1e-983f6116a7eb None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:43.560708 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc716f-7b11-48d1-8e1e-983f6116a7eb None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:43.565436 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e4459-5264-43da-a36d-ae7ef472aef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:43.566381 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e4459-5264-43da-a36d-ae7ef472aef1 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:43.566682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e4459-5264-43da-a36d-ae7ef472aef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:43.566957 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e4459-5264-43da-a36d-ae7ef472aef1 admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:43.567690 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e4459-5264-43da-a36d-ae7ef472aef1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:55:43.567993 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e4459-5264-43da-a36d-ae7ef472aef1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:43.569613 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e4459-5264-43da-a36d-ae7ef472aef1 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:43.581782 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 962/1901] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:55:43 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:55:43.586051 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e4459-5264-43da-a36d-ae7ef472aef1 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:43.596373 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e4459-5264-43da-a36d-ae7ef472aef1 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:43.602165 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47dcfbd-7871-47b4-a925-e5592d9cc5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:43.602774 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47dcfbd-7871-47b4-a925-e5592d9cc5e2 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:43.603076 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47dcfbd-7871-47b4-a925-e5592d9cc5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:43.603385 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47dcfbd-7871-47b4-a925-e5592d9cc5e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:43.604012 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47dcfbd-7871-47b4-a925-e5592d9cc5e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:43.607773 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47dcfbd-7871-47b4-a925-e5592d9cc5e2 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:43.728841 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-753029a6-8143-47e1-8787-e5d0b5ce17ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56d64a36a58e4ebb88e7f38d448b8da9'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:43.736102 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-753029a6-8143-47e1-8787-e5d0b5ce17ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56d64a36a58e4ebb88e7f38d448b8da9'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:43.736769 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-753029a6-8143-47e1-8787-e5d0b5ce17ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56d64a36a58e4ebb88e7f38d448b8da9'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:43.753311 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 963/1902] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:55:43 2022] POST /identity/v3/projects => generated 410 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:55:43.754445 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47dcfbd-7871-47b4-a925-e5592d9cc5e2 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:43.825833 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47dcfbd-7871-47b4-a925-e5592d9cc5e2 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:43.831441 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecafffd1-06c5-4789-9dac-9721047df31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:43.832062 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafffd1-06c5-4789-9dac-9721047df31c admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:43.832338 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafffd1-06c5-4789-9dac-9721047df31c admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:43.832611 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecafffd1-06c5-4789-9dac-9721047df31c admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:43.833279 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecafffd1-06c5-4789-9dac-9721047df31c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:43.835256 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-753029a6-8143-47e1-8787-e5d0b5ce17ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56d64a36a58e4ebb88e7f38d448b8da9'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:43.835389 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecafffd1-06c5-4789-9dac-9721047df31c admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:43.850820 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-753029a6-8143-47e1-8787-e5d0b5ce17ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56d64a36a58e4ebb88e7f38d448b8da9'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:43.857705 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 940/1903] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:55:43 2022] DELETE /identity/v3/users/56d64a36a58e4ebb88e7f38d448b8da9 => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:43.929702 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 964/1904] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:55:43 2022] POST /identity/v3/users => generated 306 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:43.930866 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753029a6-8143-47e1-8787-e5d0b5ce17ff admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:43.932893 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-753029a6-8143-47e1-8787-e5d0b5ce17ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:44.009155 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753029a6-8143-47e1-8787-e5d0b5ce17ff admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:44.013914 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc28d1-a9a6-4945-96da-6a6637794cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:44.014530 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc28d1-a9a6-4945-96da-6a6637794cc2 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:44.014808 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc28d1-a9a6-4945-96da-6a6637794cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:44.015293 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bc28d1-a9a6-4945-96da-6a6637794cc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:44.016015 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc28d1-a9a6-4945-96da-6a6637794cc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:44.017652 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bc28d1-a9a6-4945-96da-6a6637794cc2 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:44.027564 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 941/1905] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:43 2022] GET /identity/v3/roles => generated 1427 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:44.032380 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecafffd1-06c5-4789-9dac-9721047df31c admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:44.044171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecafffd1-06c5-4789-9dac-9721047df31c admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:44.048864 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838e922-3829-4b04-a87c-d06fc6931f26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:44.049481 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838e922-3829-4b04-a87c-d06fc6931f26 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:44.049768 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838e922-3829-4b04-a87c-d06fc6931f26 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:44.050048 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9838e922-3829-4b04-a87c-d06fc6931f26 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:44.050766 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9838e922-3829-4b04-a87c-d06fc6931f26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:44.052432 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9838e922-3829-4b04-a87c-d06fc6931f26 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:44.069495 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 965/1906] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:44 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:44.073933 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc28d1-a9a6-4945-96da-6a6637794cc2 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:44.084418 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bc28d1-a9a6-4945-96da-6a6637794cc2 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:44.090882 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fabf9-39f3-4b82-ac64-bd946ba68ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:44.092063 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fabf9-39f3-4b82-ac64-bd946ba68ed2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:44.092614 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fabf9-39f3-4b82-ac64-bd946ba68ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:44.092900 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fabf9-39f3-4b82-ac64-bd946ba68ed2 admin admin] PATH_INFO: `/v3/projects/772ace22947e4a9893fd7c87ccb22905/users/c584d40a93104884b26a2b4b73bdcd2b/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:44.113093 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88fabf9-39f3-4b82-ac64-bd946ba68ed2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=772ace22947e4a9893fd7c87ccb22905, user_id=c584d40a93104884b26a2b4b73bdcd2b, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:44.115935 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88fabf9-39f3-4b82-ac64-bd946ba68ed2 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:44.220818 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838e922-3829-4b04-a87c-d06fc6931f26 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:44.222841 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-9838e922-3829-4b04-a87c-d06fc6931f26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:44.242371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 942/1907] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:44 2022] PUT /identity/v3/projects/772ace22947e4a9893fd7c87ccb22905/users/c584d40a93104884b26a2b4b73bdcd2b/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:44.258327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd69eed6-aa21-482c-9a61-bd0459972c5c None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:44.258702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd69eed6-aa21-482c-9a61-bd0459972c5c None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:44.258866 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd69eed6-aa21-482c-9a61-bd0459972c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:44.261093 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9838e922-3829-4b04-a87c-d06fc6931f26 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:44.291836 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-fd69eed6-aa21-482c-9a61-bd0459972c5c None None] MFA Rules not processed for user `c584d40a93104884b26a2b4b73bdcd2b`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:44.293127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-fd69eed6-aa21-482c-9a61-bd0459972c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:44.366957 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7fcf2-b19f-46a1-8f69-bc6ffda68c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:44.367390 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7fcf2-b19f-46a1-8f69-bc6ffda68c4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:44.367390 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7fcf2-b19f-46a1-8f69-bc6ffda68c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:44.367891 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7fcf2-b19f-46a1-8f69-bc6ffda68c4d admin admin] PATH_INFO: `/v3/users/a6ac0e65fe8d4ba7aeedc6b424379a7c` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:44.369011 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7fcf2-b19f-46a1-8f69-bc6ffda68c4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6ac0e65fe8d4ba7aeedc6b424379a7c)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:44.371519 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7fcf2-b19f-46a1-8f69-bc6ffda68c4d admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:44.377213 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 943/1908] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:44 2022] POST /identity/v3/auth/tokens => generated 3296 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:55:44.378358 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd69eed6-aa21-482c-9a61-bd0459972c5c None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:44.382731 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-fd69eed6-aa21-482c-9a61-bd0459972c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:44.400320 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd69eed6-aa21-482c-9a61-bd0459972c5c None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:44.406738 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da370733-8720-4bd3-91aa-d342fcd7d445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:44.408018 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da370733-8720-4bd3-91aa-d342fcd7d445 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:44.408513 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da370733-8720-4bd3-91aa-d342fcd7d445 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:44.409029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da370733-8720-4bd3-91aa-d342fcd7d445 admin admin] PATH_INFO: `/v3/projects/171c9eabe04846809b6d2696aa09085d` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:44.410734 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da370733-8720-4bd3-91aa-d342fcd7d445 admin admin] RBAC: Authorizing `identity:delete_project(project_id=171c9eabe04846809b6d2696aa09085d)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:44.414490 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da370733-8720-4bd3-91aa-d342fcd7d445 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:44.642887 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-da370733-8720-4bd3-91aa-d342fcd7d445 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '171c9eabe04846809b6d2696aa09085d', 'request_id': 'req-da370733-8720-4bd3-91aa-d342fcd7d445'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:44.652202 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-75c7fcf2-b19f-46a1-8f69-bc6ffda68c4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6ac0e65fe8d4ba7aeedc6b424379a7c'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:44.653390 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-75c7fcf2-b19f-46a1-8f69-bc6ffda68c4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6ac0e65fe8d4ba7aeedc6b424379a7c'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:44.767422 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-da370733-8720-4bd3-91aa-d342fcd7d445 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '171c9eabe04846809b6d2696aa09085d', 'request_id': 'req-da370733-8720-4bd3-91aa-d342fcd7d445'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:44.775126 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-da370733-8720-4bd3-91aa-d342fcd7d445 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '171c9eabe04846809b6d2696aa09085d', 'request_id': 'req-da370733-8720-4bd3-91aa-d342fcd7d445'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:44.788241 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-da370733-8720-4bd3-91aa-d342fcd7d445 admin admin] The token cache is being invalidate because project 171c9eabe04846809b6d2696aa09085d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:55:44.788550 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-da370733-8720-4bd3-91aa-d342fcd7d445 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:44.791464 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-da370733-8720-4bd3-91aa-d342fcd7d445 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:44.793263 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 944/1909] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:55:44 2022] DELETE /identity/v3/projects/171c9eabe04846809b6d2696aa09085d => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:44.796814 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:44.798017 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da370733-8720-4bd3-91aa-d342fcd7d445 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:44.799426 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-da370733-8720-4bd3-91aa-d342fcd7d445 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:44.882087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da370733-8720-4bd3-91aa-d342fcd7d445 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:44.888167 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c53af8-8798-4489-8e8e-914c9e9a16a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:44.888759 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c53af8-8798-4489-8e8e-914c9e9a16a0 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:44.889044 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c53af8-8798-4489-8e8e-914c9e9a16a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:44.889325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c53af8-8798-4489-8e8e-914c9e9a16a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:44.892483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-75c7fcf2-b19f-46a1-8f69-bc6ffda68c4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6ac0e65fe8d4ba7aeedc6b424379a7c'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:44.896964 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c53af8-8798-4489-8e8e-914c9e9a16a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:44.899071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c53af8-8798-4489-8e8e-914c9e9a16a0 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:44.903965 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-75c7fcf2-b19f-46a1-8f69-bc6ffda68c4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6ac0e65fe8d4ba7aeedc6b424379a7c'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:44.919145 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-75c7fcf2-b19f-46a1-8f69-bc6ffda68c4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6ac0e65fe8d4ba7aeedc6b424379a7c'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:44.929089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 966/1910] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:55:44 2022] DELETE /identity/v3/users/a6ac0e65fe8d4ba7aeedc6b424379a7c => generated 0 bytes in 706 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:55:44.930632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573f98f-5d40-4250-b948-f64f41f532ce None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:44.931202 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573f98f-5d40-4250-b948-f64f41f532ce None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:44.931314 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6573f98f-5d40-4250-b948-f64f41f532ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:44.953843 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-6573f98f-5d40-4250-b948-f64f41f532ce None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:44.956218 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-6573f98f-5d40-4250-b948-f64f41f532ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:45.015785 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 945/1911] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:55:44 2022] POST /identity/v3/projects => generated 410 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:45.018088 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b36c5-95b7-4175-a595-1ef79e0e3181 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:45.018733 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b36c5-95b7-4175-a595-1ef79e0e3181 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:45.019334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653b36c5-95b7-4175-a595-1ef79e0e3181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:45.062894 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-653b36c5-95b7-4175-a595-1ef79e0e3181 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:45.063511 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 967/1912] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:44 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:45.065436 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573f98f-5d40-4250-b948-f64f41f532ce None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:45.067108 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-6573f98f-5d40-4250-b948-f64f41f532ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:45.067945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-653b36c5-95b7-4175-a595-1ef79e0e3181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:45.082495 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6573f98f-5d40-4250-b948-f64f41f532ce None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:45.151293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5106363-4aec-4953-a5e8-3eac5882bead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:45.152014 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5106363-4aec-4953-a5e8-3eac5882bead admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:45.152306 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5106363-4aec-4953-a5e8-3eac5882bead admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:45.152612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5106363-4aec-4953-a5e8-3eac5882bead admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:45.153333 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5106363-4aec-4953-a5e8-3eac5882bead admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:45.155449 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5106363-4aec-4953-a5e8-3eac5882bead admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:45.175105 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:45.176786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 946/1913] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:45 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:45.178866 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5e530-9b3c-4617-a9fa-15b6539c0386 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:45.179206 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5e530-9b3c-4617-a9fa-15b6539c0386 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:45.179530 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b5e530-9b3c-4617-a9fa-15b6539c0386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:45.203080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-29b5e530-9b3c-4617-a9fa-15b6539c0386 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:45.204320 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-29b5e530-9b3c-4617-a9fa-15b6539c0386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:45.220643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 947/1914] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:45 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:45.223298 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b5e530-9b3c-4617-a9fa-15b6539c0386 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:45.233345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b5e530-9b3c-4617-a9fa-15b6539c0386 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:45.248061 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3a771-1106-410d-a20e-9f64ae416f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:45.248961 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3a771-1106-410d-a20e-9f64ae416f6e admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:45.249431 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3a771-1106-410d-a20e-9f64ae416f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:45.250979 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3a771-1106-410d-a20e-9f64ae416f6e admin admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:45.251256 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3a771-1106-410d-a20e-9f64ae416f6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:55:45.251834 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3a771-1106-410d-a20e-9f64ae416f6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:45.253881 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3a771-1106-410d-a20e-9f64ae416f6e admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:45.253997 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 968/1915] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:55:45 2022] POST /identity/v3/users => generated 312 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:45.254922 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5106363-4aec-4953-a5e8-3eac5882bead admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:45.259567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-f5106363-4aec-4953-a5e8-3eac5882bead admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:45.273334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 948/1916] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:55:45 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:55:45.275180 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3a771-1106-410d-a20e-9f64ae416f6e admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:45.294397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3a771-1106-410d-a20e-9f64ae416f6e admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:45.316964 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a6902-b7a5-4044-8a94-897536517411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:45.318117 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a6902-b7a5-4044-8a94-897536517411 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:45.318805 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a6902-b7a5-4044-8a94-897536517411 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:45.319323 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4a6902-b7a5-4044-8a94-897536517411 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:45.320302 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4a6902-b7a5-4044-8a94-897536517411 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:45.322456 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4a6902-b7a5-4044-8a94-897536517411 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:45.337009 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5106363-4aec-4953-a5e8-3eac5882bead admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:45.337940 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 949/1917] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:45 2022] GET /identity/v3/roles => generated 1427 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:45.339161 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a6902-b7a5-4044-8a94-897536517411 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:45.345765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c5f79-d871-4343-a85d-cd2cf9641811 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:45.347073 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c5f79-d871-4343-a85d-cd2cf9641811 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:45.347415 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c5f79-d871-4343-a85d-cd2cf9641811 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:45.347679 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649c5f79-d871-4343-a85d-cd2cf9641811 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:45.357770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4a6902-b7a5-4044-8a94-897536517411 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:45.365725 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649c5f79-d871-4343-a85d-cd2cf9641811 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:45.369623 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-649c5f79-d871-4343-a85d-cd2cf9641811 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:45.379933 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f06e65-c7c2-44ac-a5dc-2e65665f3109 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:45.380244 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 969/1918] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:45 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:45.381629 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f06e65-c7c2-44ac-a5dc-2e65665f3109 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:45.381744 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c5f79-d871-4343-a85d-cd2cf9641811 service neutron] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:45.383453 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f06e65-c7c2-44ac-a5dc-2e65665f3109 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:45.384093 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f06e65-c7c2-44ac-a5dc-2e65665f3109 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:45.385049 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f06e65-c7c2-44ac-a5dc-2e65665f3109 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:45.390777 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f06e65-c7c2-44ac-a5dc-2e65665f3109 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:45.394160 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-649c5f79-d871-4343-a85d-cd2cf9641811 service neutron] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:45.400150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa251d-5b92-4281-a6a0-33d8030609e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:45.400806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa251d-5b92-4281-a6a0-33d8030609e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:45.401094 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa251d-5b92-4281-a6a0-33d8030609e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:45.401375 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eaa251d-5b92-4281-a6a0-33d8030609e2 admin admin] PATH_INFO: `/v3/projects/6da1fab8502c40b4b5edccb5e97e77c2/users/40546cb4010f4163b5c3374b09e55996/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:45.424262 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaa251d-5b92-4281-a6a0-33d8030609e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6da1fab8502c40b4b5edccb5e97e77c2, user_id=40546cb4010f4163b5c3374b09e55996, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:45.428018 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eaa251d-5b92-4281-a6a0-33d8030609e2 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:45.512519 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 950/1919] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:55:45 2022] POST /identity/v3/projects => generated 386 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:45.513464 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 970/1920] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:45 2022] PUT /identity/v3/projects/6da1fab8502c40b4b5edccb5e97e77c2/users/40546cb4010f4163b5c3374b09e55996/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:45.518284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f06e65-c7c2-44ac-a5dc-2e65665f3109 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:45.518884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa251d-5b92-4281-a6a0-33d8030609e2 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:45.619792 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f06e65-c7c2-44ac-a5dc-2e65665f3109 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:45.627787 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc05d25-2ec6-4790-a3d6-65c44171a42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:45.629147 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc05d25-2ec6-4790-a3d6-65c44171a42f admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:45.629430 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc05d25-2ec6-4790-a3d6-65c44171a42f admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:45.629947 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc05d25-2ec6-4790-a3d6-65c44171a42f admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:45.631061 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc05d25-2ec6-4790-a3d6-65c44171a42f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:45.633725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc05d25-2ec6-4790-a3d6-65c44171a42f admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:45.633983 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eaa251d-5b92-4281-a6a0-33d8030609e2 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:45.640469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8973ba7-065e-4eef-ad3f-d52fbd65f94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:45.641552 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8973ba7-065e-4eef-ad3f-d52fbd65f94c admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:45.641952 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8973ba7-065e-4eef-ad3f-d52fbd65f94c admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:45.642338 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8973ba7-065e-4eef-ad3f-d52fbd65f94c admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:45.643222 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8973ba7-065e-4eef-ad3f-d52fbd65f94c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:45.645447 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8973ba7-065e-4eef-ad3f-d52fbd65f94c admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:45.658479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 971/1921] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:45 2022] GET /identity/v3/roles => generated 1427 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:45.662628 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8973ba7-065e-4eef-ad3f-d52fbd65f94c admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:45.674367 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8973ba7-065e-4eef-ad3f-d52fbd65f94c admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:45.679697 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45635cb2-ca95-4af1-8ce3-3bd472383d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:45.680557 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45635cb2-ca95-4af1-8ce3-3bd472383d1e admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:45.680886 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45635cb2-ca95-4af1-8ce3-3bd472383d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:45.681111 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45635cb2-ca95-4af1-8ce3-3bd472383d1e admin admin] PATH_INFO: `/v3/projects/6da1fab8502c40b4b5edccb5e97e77c2/users/40546cb4010f4163b5c3374b09e55996/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:45.685103 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45635cb2-ca95-4af1-8ce3-3bd472383d1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6da1fab8502c40b4b5edccb5e97e77c2, user_id=40546cb4010f4163b5c3374b09e55996, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:45.687860 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45635cb2-ca95-4af1-8ce3-3bd472383d1e admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:45.699397 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 972/1922] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:45 2022] PUT /identity/v3/projects/6da1fab8502c40b4b5edccb5e97e77c2/users/40546cb4010f4163b5c3374b09e55996/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:45.713367 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac673d9-38d0-4122-a0ae-cb6ad52cee24 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:45.713951 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac673d9-38d0-4122-a0ae-cb6ad52cee24 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:45.714243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac673d9-38d0-4122-a0ae-cb6ad52cee24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:45.752372 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 951/1923] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:55:45 2022] POST /identity/v3/users => generated 294 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:45.766049 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-eac673d9-38d0-4122-a0ae-cb6ad52cee24 None None] MFA Rules not processed for user `40546cb4010f4163b5c3374b09e55996`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:45.766049 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-eac673d9-38d0-4122-a0ae-cb6ad52cee24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:45.768563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc05d25-2ec6-4790-a3d6-65c44171a42f admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:45.783745 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc05d25-2ec6-4790-a3d6-65c44171a42f admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:45.872599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 973/1924] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:45 2022] POST /identity/v3/auth/tokens => generated 3363 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:55:45.876188 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c35d3-4d2a-4f98-b382-5d3607e2988e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:45.876188 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c35d3-4d2a-4f98-b382-5d3607e2988e admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:45.876484 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c35d3-4d2a-4f98-b382-5d3607e2988e admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:45.876830 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c35d3-4d2a-4f98-b382-5d3607e2988e admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:45.877770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c35d3-4d2a-4f98-b382-5d3607e2988e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:45.879550 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c35d3-4d2a-4f98-b382-5d3607e2988e admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:45.883275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac673d9-38d0-4122-a0ae-cb6ad52cee24 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:45.885278 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-eac673d9-38d0-4122-a0ae-cb6ad52cee24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:45.899410 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 952/1925] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:45 2022] GET /identity/v3/roles => generated 1427 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:45.900530 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c35d3-4d2a-4f98-b382-5d3607e2988e admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:45.915477 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c35d3-4d2a-4f98-b382-5d3607e2988e admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:45.920914 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7856bd-6d1f-4b8c-9298-cb5622f494df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:45.921564 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7856bd-6d1f-4b8c-9298-cb5622f494df admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:45.921849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7856bd-6d1f-4b8c-9298-cb5622f494df admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:45.922127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7856bd-6d1f-4b8c-9298-cb5622f494df admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:45.923144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7856bd-6d1f-4b8c-9298-cb5622f494df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:45.924946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7856bd-6d1f-4b8c-9298-cb5622f494df admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:45.937129 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 953/1926] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:45 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:45.943455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7856bd-6d1f-4b8c-9298-cb5622f494df admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:45.955063 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7856bd-6d1f-4b8c-9298-cb5622f494df admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:45.960763 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fdcbf1-6a74-4726-904a-ae91f5f857a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:45.961237 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fdcbf1-6a74-4726-904a-ae91f5f857a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:45.961617 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fdcbf1-6a74-4726-904a-ae91f5f857a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:45.962062 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fdcbf1-6a74-4726-904a-ae91f5f857a0 admin admin] PATH_INFO: `/v3/projects/f48684fc4aad4857b70e9a2a7d5582cd/users/600cc6667bcb48539029bc494627728a/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:45.964512 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac673d9-38d0-4122-a0ae-cb6ad52cee24 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:45.969081 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446fc6a6-95a4-4b15-9bf1-ced0b2522e35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:45.971474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fc6a6-95a4-4b15-9bf1-ced0b2522e35 service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:45.971474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fc6a6-95a4-4b15-9bf1-ced0b2522e35 service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:45.971474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446fc6a6-95a4-4b15-9bf1-ced0b2522e35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:45.980157 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fdcbf1-6a74-4726-904a-ae91f5f857a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f48684fc4aad4857b70e9a2a7d5582cd, user_id=600cc6667bcb48539029bc494627728a, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:45.983534 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fdcbf1-6a74-4726-904a-ae91f5f857a0 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:45.983997 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446fc6a6-95a4-4b15-9bf1-ced0b2522e35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:45.986274 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446fc6a6-95a4-4b15-9bf1-ced0b2522e35 service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:45.999514 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 974/1927] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:45 2022] GET /identity/v3/auth/tokens => generated 3363 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:55:46.133729 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 954/1928] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:45 2022] PUT /identity/v3/projects/f48684fc4aad4857b70e9a2a7d5582cd/users/600cc6667bcb48539029bc494627728a/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:55:46.144863 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa640c35-e3dc-4942-a2fa-5c906604ec0d None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:46.145321 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa640c35-e3dc-4942-a2fa-5c906604ec0d None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:46.145456 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa640c35-e3dc-4942-a2fa-5c906604ec0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:46.176307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-aa640c35-e3dc-4942-a2fa-5c906604ec0d None None] MFA Rules not processed for user `600cc6667bcb48539029bc494627728a`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:46.177538 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-aa640c35-e3dc-4942-a2fa-5c906604ec0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:46.249113 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 975/1929] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:46 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:46.254319 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fdcbf1-6a74-4726-904a-ae91f5f857a0 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:46.343098 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fdcbf1-6a74-4726-904a-ae91f5f857a0 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:46.349616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ed260-40ba-461d-91e4-ccc8a7281d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:46.349616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ed260-40ba-461d-91e4-ccc8a7281d00 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:46.349616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ed260-40ba-461d-91e4-ccc8a7281d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:46.349616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168ed260-40ba-461d-91e4-ccc8a7281d00 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:46.355914 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ed260-40ba-461d-91e4-ccc8a7281d00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:46.355914 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168ed260-40ba-461d-91e4-ccc8a7281d00 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:46.488871 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 955/1930] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:55:46 2022] POST /identity/v3/projects => generated 386 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:46.488999 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa640c35-e3dc-4942-a2fa-5c906604ec0d None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:46.508079 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa640c35-e3dc-4942-a2fa-5c906604ec0d None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:46.509846 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ed260-40ba-461d-91e4-ccc8a7281d00 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:46.568567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:46.623681 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168ed260-40ba-461d-91e4-ccc8a7281d00 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:46.629677 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc748cc2-ca80-445a-b8f9-0dc41df9facd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:46.631618 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc748cc2-ca80-445a-b8f9-0dc41df9facd service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:46.632195 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc748cc2-ca80-445a-b8f9-0dc41df9facd service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:46.633138 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc748cc2-ca80-445a-b8f9-0dc41df9facd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:46.636036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-bc748cc2-ca80-445a-b8f9-0dc41df9facd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:46.636673 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32874e3-2e35-4918-809f-2100cbccd082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:46.637650 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32874e3-2e35-4918-809f-2100cbccd082 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:46.638124 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32874e3-2e35-4918-809f-2100cbccd082 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:46.641981 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32874e3-2e35-4918-809f-2100cbccd082 admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:46.641981 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32874e3-2e35-4918-809f-2100cbccd082 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:46.641981 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32874e3-2e35-4918-809f-2100cbccd082 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:46.680755 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:46.722734 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc748cc2-ca80-445a-b8f9-0dc41df9facd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:46.724454 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc748cc2-ca80-445a-b8f9-0dc41df9facd service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:46.727237 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 976/1931] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:55:46 2022] POST /identity/v3/users => generated 300 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:46.727934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32874e3-2e35-4918-809f-2100cbccd082 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:46.734820 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-b32874e3-2e35-4918-809f-2100cbccd082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:46.741482 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 956/1932] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:46 2022] GET /identity/v3/auth/tokens => generated 3296 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:55:46.742539 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc748cc2-ca80-445a-b8f9-0dc41df9facd service cinder] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:46.748630 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32874e3-2e35-4918-809f-2100cbccd082 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:46.758288 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc748cc2-ca80-445a-b8f9-0dc41df9facd service cinder] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:46.759500 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f365ef-54a8-4908-83f2-53c9783f071c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:46.760321 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f365ef-54a8-4908-83f2-53c9783f071c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:46.763478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f365ef-54a8-4908-83f2-53c9783f071c admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:46.768846 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f365ef-54a8-4908-83f2-53c9783f071c admin admin] PATH_INFO: `/v3/projects/a5e86f7861604d8c93f0737c4303f279` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:46.770182 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f365ef-54a8-4908-83f2-53c9783f071c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a5e86f7861604d8c93f0737c4303f279)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:46.774459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd9133-968e-4b07-be55-487a929398e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:46.776517 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd9133-968e-4b07-be55-487a929398e0 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:46.776904 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd9133-968e-4b07-be55-487a929398e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:46.777929 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd9133-968e-4b07-be55-487a929398e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:46.778431 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f365ef-54a8-4908-83f2-53c9783f071c admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:46.783260 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fd9133-968e-4b07-be55-487a929398e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:46.795649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fd9133-968e-4b07-be55-487a929398e0 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:46.836436 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 957/1933] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:46 2022] GET /identity/v3/roles => generated 1427 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:46.838532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd9133-968e-4b07-be55-487a929398e0 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:46.857257 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd9133-968e-4b07-be55-487a929398e0 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:46.868675 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071533c6-4915-4ea0-95f5-1739ecba46c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:46.868675 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071533c6-4915-4ea0-95f5-1739ecba46c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:46.868675 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071533c6-4915-4ea0-95f5-1739ecba46c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:46.868675 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071533c6-4915-4ea0-95f5-1739ecba46c2 admin admin] PATH_INFO: `/v3/projects/37f86be5f17f4a3dba854033f38f5745/users/ce4d1a24e763414fb13a2a846e926ce1/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:46.890162 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071533c6-4915-4ea0-95f5-1739ecba46c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37f86be5f17f4a3dba854033f38f5745, user_id=ce4d1a24e763414fb13a2a846e926ce1, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:46.894491 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071533c6-4915-4ea0-95f5-1739ecba46c2 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:47.019890 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 958/1934] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:46 2022] PUT /identity/v3/projects/37f86be5f17f4a3dba854033f38f5745/users/ce4d1a24e763414fb13a2a846e926ce1/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:47.035748 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071533c6-4915-4ea0-95f5-1739ecba46c2 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:47.057257 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-64f365ef-54a8-4908-83f2-53c9783f071c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a5e86f7861604d8c93f0737c4303f279', 'request_id': 'req-64f365ef-54a8-4908-83f2-53c9783f071c'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:47.065047 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:47.078275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-64f365ef-54a8-4908-83f2-53c9783f071c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a5e86f7861604d8c93f0737c4303f279', 'request_id': 'req-64f365ef-54a8-4908-83f2-53c9783f071c'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:47.143838 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-64f365ef-54a8-4908-83f2-53c9783f071c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a5e86f7861604d8c93f0737c4303f279', 'request_id': 'req-64f365ef-54a8-4908-83f2-53c9783f071c'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:47.158086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-64f365ef-54a8-4908-83f2-53c9783f071c admin admin] The token cache is being invalidate because project a5e86f7861604d8c93f0737c4303f279 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:55:47.159565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-64f365ef-54a8-4908-83f2-53c9783f071c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:47.160853 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-64f365ef-54a8-4908-83f2-53c9783f071c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:47.163289 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 977/1935] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:55:46 2022] DELETE /identity/v3/projects/a5e86f7861604d8c93f0737c4303f279 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:47.167625 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f365ef-54a8-4908-83f2-53c9783f071c admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:47.169661 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-64f365ef-54a8-4908-83f2-53c9783f071c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:47.190667 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071533c6-4915-4ea0-95f5-1739ecba46c2 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:47.285599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f365ef-54a8-4908-83f2-53c9783f071c admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:47.289559 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe08590-9b2b-43b1-b767-4f5b2c8b5055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:47.290374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe08590-9b2b-43b1-b767-4f5b2c8b5055 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:47.290823 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe08590-9b2b-43b1-b767-4f5b2c8b5055 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:47.291150 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe08590-9b2b-43b1-b767-4f5b2c8b5055 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:47.292553 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe08590-9b2b-43b1-b767-4f5b2c8b5055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:47.295561 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe08590-9b2b-43b1-b767-4f5b2c8b5055 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:47.302573 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738ea85c-f880-4b9c-a26a-dc2d7a1ecab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:47.304273 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ea85c-f880-4b9c-a26a-dc2d7a1ecab3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:47.305860 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ea85c-f880-4b9c-a26a-dc2d7a1ecab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:47.306141 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738ea85c-f880-4b9c-a26a-dc2d7a1ecab3 admin admin] PATH_INFO: `/v3/users/31c78db97916413ab8b6b15b13ee7e1b` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:47.309714 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738ea85c-f880-4b9c-a26a-dc2d7a1ecab3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31c78db97916413ab8b6b15b13ee7e1b)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:47.311795 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738ea85c-f880-4b9c-a26a-dc2d7a1ecab3 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:47.318200 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 959/1936] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:47 2022] GET /identity/v3/roles => generated 1427 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:47.319027 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe08590-9b2b-43b1-b767-4f5b2c8b5055 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:47.320820 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-cfe08590-9b2b-43b1-b767-4f5b2c8b5055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:47.340231 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe08590-9b2b-43b1-b767-4f5b2c8b5055 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:47.348712 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2beb673-c511-4eb8-a29d-807fede26fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:47.349476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2beb673-c511-4eb8-a29d-807fede26fd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:47.349821 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2beb673-c511-4eb8-a29d-807fede26fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:47.350201 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2beb673-c511-4eb8-a29d-807fede26fd0 admin admin] PATH_INFO: `/v3/projects/37f86be5f17f4a3dba854033f38f5745/users/ce4d1a24e763414fb13a2a846e926ce1/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:47.352265 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2beb673-c511-4eb8-a29d-807fede26fd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37f86be5f17f4a3dba854033f38f5745, user_id=ce4d1a24e763414fb13a2a846e926ce1, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:47.355516 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2beb673-c511-4eb8-a29d-807fede26fd0 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:47.371701 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 960/1937] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:47 2022] PUT /identity/v3/projects/37f86be5f17f4a3dba854033f38f5745/users/ce4d1a24e763414fb13a2a846e926ce1/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:47.383335 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777954a2-3506-4ef1-a6b8-f28b3dfcdd84 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:47.383752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777954a2-3506-4ef1-a6b8-f28b3dfcdd84 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:47.384122 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777954a2-3506-4ef1-a6b8-f28b3dfcdd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:47.470083 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-777954a2-3506-4ef1-a6b8-f28b3dfcdd84 None None] MFA Rules not processed for user `ce4d1a24e763414fb13a2a846e926ce1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:47.471352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-777954a2-3506-4ef1-a6b8-f28b3dfcdd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:47.584066 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-738ea85c-f880-4b9c-a26a-dc2d7a1ecab3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31c78db97916413ab8b6b15b13ee7e1b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:47.584736 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-738ea85c-f880-4b9c-a26a-dc2d7a1ecab3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31c78db97916413ab8b6b15b13ee7e1b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:47.607116 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 961/1938] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:47 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:47.623757 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777954a2-3506-4ef1-a6b8-f28b3dfcdd84 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:47.625766 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-777954a2-3506-4ef1-a6b8-f28b3dfcdd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:47.703864 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-738ea85c-f880-4b9c-a26a-dc2d7a1ecab3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31c78db97916413ab8b6b15b13ee7e1b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:47.715642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-738ea85c-f880-4b9c-a26a-dc2d7a1ecab3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31c78db97916413ab8b6b15b13ee7e1b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:47.718558 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777954a2-3506-4ef1-a6b8-f28b3dfcdd84 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:47.724820 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d6248-d62a-439a-a4a2-d84fbf29393a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:47.725703 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d6248-d62a-439a-a4a2-d84fbf29393a service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:47.726269 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d6248-d62a-439a-a4a2-d84fbf29393a service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:47.727248 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6d6248-d62a-439a-a4a2-d84fbf29393a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:47.740240 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d6248-d62a-439a-a4a2-d84fbf29393a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:47.741614 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-738ea85c-f880-4b9c-a26a-dc2d7a1ecab3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31c78db97916413ab8b6b15b13ee7e1b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:47.743346 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6d6248-d62a-439a-a4a2-d84fbf29393a service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:47.755134 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 962/1939] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:47 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:47.761288 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 978/1940] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:55:47 2022] DELETE /identity/v3/users/31c78db97916413ab8b6b15b13ee7e1b => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:47.837256 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:48.709745 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:49.061827 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d6248-d62a-439a-a4a2-d84fbf29393a service cinder] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:49.194572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6d6248-d62a-439a-a4a2-d84fbf29393a service cinder] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:49.294827 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b7c596-8304-45df-af35-971c2165cdc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:49.295964 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7c596-8304-45df-af35-971c2165cdc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:49.296395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7c596-8304-45df-af35-971c2165cdc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:49.296822 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b7c596-8304-45df-af35-971c2165cdc9 admin admin] PATH_INFO: `/v3/projects/8e3aa72e1e494f98b278edddee59f86d` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:49.298363 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b7c596-8304-45df-af35-971c2165cdc9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e3aa72e1e494f98b278edddee59f86d)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:49.300708 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b7c596-8304-45df-af35-971c2165cdc9 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:49.438468 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-89b7c596-8304-45df-af35-971c2165cdc9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e3aa72e1e494f98b278edddee59f86d', 'request_id': 'req-89b7c596-8304-45df-af35-971c2165cdc9'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:49.487105 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:49.510651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-89b7c596-8304-45df-af35-971c2165cdc9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e3aa72e1e494f98b278edddee59f86d', 'request_id': 'req-89b7c596-8304-45df-af35-971c2165cdc9'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:49.521536 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-89b7c596-8304-45df-af35-971c2165cdc9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e3aa72e1e494f98b278edddee59f86d', 'request_id': 'req-89b7c596-8304-45df-af35-971c2165cdc9'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:49.539812 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-89b7c596-8304-45df-af35-971c2165cdc9 admin admin] The token cache is being invalidate because project 8e3aa72e1e494f98b278edddee59f86d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:55:49.540116 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-89b7c596-8304-45df-af35-971c2165cdc9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:49.540815 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-89b7c596-8304-45df-af35-971c2165cdc9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:49.542541 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 963/1941] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:55:49 2022] DELETE /identity/v3/projects/8e3aa72e1e494f98b278edddee59f86d => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:49.548472 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280edea-9eff-42fe-9650-75b77c75a358 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:49.555241 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280edea-9eff-42fe-9650-75b77c75a358 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:49.555241 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d280edea-9eff-42fe-9650-75b77c75a358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:49.586588 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-d280edea-9eff-42fe-9650-75b77c75a358 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:49.588007 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-d280edea-9eff-42fe-9650-75b77c75a358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:49.682893 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 979/1942] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:49 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:49.691200 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da970f-a901-4447-8399-d7a38d3e4e43 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:49.691489 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da970f-a901-4447-8399-d7a38d3e4e43 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:49.691775 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74da970f-a901-4447-8399-d7a38d3e4e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:49.712270 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-74da970f-a901-4447-8399-d7a38d3e4e43 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:49.713525 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-74da970f-a901-4447-8399-d7a38d3e4e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:49.725093 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 964/1943] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:49 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:49.729089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280edea-9eff-42fe-9650-75b77c75a358 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:49.740011 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d280edea-9eff-42fe-9650-75b77c75a358 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:49.745153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1bcc9-8eba-4f38-9d23-5d6b11e5eda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:49.745771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1bcc9-8eba-4f38-9d23-5d6b11e5eda0 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:49.746049 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1bcc9-8eba-4f38-9d23-5d6b11e5eda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:49.746494 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1bcc9-8eba-4f38-9d23-5d6b11e5eda0 admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:49.747600 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff1bcc9-8eba-4f38-9d23-5d6b11e5eda0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:55:49.747909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff1bcc9-8eba-4f38-9d23-5d6b11e5eda0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:49.749506 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff1bcc9-8eba-4f38-9d23-5d6b11e5eda0 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:49.766976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 980/1944] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:55:49 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:55:49.768592 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da970f-a901-4447-8399-d7a38d3e4e43 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:49.782216 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74da970f-a901-4447-8399-d7a38d3e4e43 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:49.788216 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eba0c8-8e0f-42a4-9fa3-0f87ebaac546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:49.788932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eba0c8-8e0f-42a4-9fa3-0f87ebaac546 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:49.789266 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eba0c8-8e0f-42a4-9fa3-0f87ebaac546 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:49.789596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eba0c8-8e0f-42a4-9fa3-0f87ebaac546 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:49.794322 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eba0c8-8e0f-42a4-9fa3-0f87ebaac546 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:49.794322 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eba0c8-8e0f-42a4-9fa3-0f87ebaac546 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:49.939026 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 965/1945] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:55:49 2022] POST /identity/v3/projects => generated 378 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:49.940256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1bcc9-8eba-4f38-9d23-5d6b11e5eda0 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:49.957176 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1bcc9-8eba-4f38-9d23-5d6b11e5eda0 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:50.096235 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7722355-32c5-4dd9-b510-6b28f832032f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:50.097160 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7722355-32c5-4dd9-b510-6b28f832032f admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:50.097611 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7722355-32c5-4dd9-b510-6b28f832032f admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:50.098089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7722355-32c5-4dd9-b510-6b28f832032f admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:50.098749 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7722355-32c5-4dd9-b510-6b28f832032f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:50.100734 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7722355-32c5-4dd9-b510-6b28f832032f admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:50.202397 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 981/1946] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:55:49 2022] POST /identity/v3/users => generated 290 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:50.206508 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eba0c8-8e0f-42a4-9fa3-0f87ebaac546 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:50.246030 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eba0c8-8e0f-42a4-9fa3-0f87ebaac546 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:50.265809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a59ba-68cb-40a1-8f5e-a4bc3963e021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:50.267005 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a59ba-68cb-40a1-8f5e-a4bc3963e021 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:50.274743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a59ba-68cb-40a1-8f5e-a4bc3963e021 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:50.275753 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319a59ba-68cb-40a1-8f5e-a4bc3963e021 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:50.276978 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a59ba-68cb-40a1-8f5e-a4bc3963e021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:50.280002 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319a59ba-68cb-40a1-8f5e-a4bc3963e021 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:50.301283 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 966/1947] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:50 2022] GET /identity/v3/roles => generated 1427 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:50.301671 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7722355-32c5-4dd9-b510-6b28f832032f admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:50.312703 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7722355-32c5-4dd9-b510-6b28f832032f admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:50.317847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f33019-429c-46a0-9bc2-83be8e754bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:50.318229 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f33019-429c-46a0-9bc2-83be8e754bf7 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:50.318512 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f33019-429c-46a0-9bc2-83be8e754bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:50.318796 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f33019-429c-46a0-9bc2-83be8e754bf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:50.319533 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f33019-429c-46a0-9bc2-83be8e754bf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:50.321275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f33019-429c-46a0-9bc2-83be8e754bf7 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:50.363708 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 982/1948] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:50 2022] GET /identity/v3/roles => generated 1427 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:50.367282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a59ba-68cb-40a1-8f5e-a4bc3963e021 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:50.383072 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319a59ba-68cb-40a1-8f5e-a4bc3963e021 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:50.389446 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d1824d-5c6e-4b89-bb82-dc087b8a9dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:50.390311 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1824d-5c6e-4b89-bb82-dc087b8a9dcd admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:50.397944 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1824d-5c6e-4b89-bb82-dc087b8a9dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:50.398414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d1824d-5c6e-4b89-bb82-dc087b8a9dcd admin admin] PATH_INFO: `/v3/projects/0ba41428b84447e99a7f0f7f143c7f90/users/ed33ed788b5d47ffa82ff121f7935cb2/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:50.420678 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d1824d-5c6e-4b89-bb82-dc087b8a9dcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ba41428b84447e99a7f0f7f143c7f90, user_id=ed33ed788b5d47ffa82ff121f7935cb2, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:50.425107 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d1824d-5c6e-4b89-bb82-dc087b8a9dcd admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:50.540592 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:50.545695 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 967/1949] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:50 2022] PUT /identity/v3/projects/0ba41428b84447e99a7f0f7f143c7f90/users/ed33ed788b5d47ffa82ff121f7935cb2/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:50.561119 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f33019-429c-46a0-9bc2-83be8e754bf7 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:50.563075 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2d7fe-e92a-4f9b-9cef-45f5c4706946 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:50.563799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2d7fe-e92a-4f9b-9cef-45f5c4706946 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:50.564148 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2d7fe-e92a-4f9b-9cef-45f5c4706946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:50.565967 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-80f33019-429c-46a0-9bc2-83be8e754bf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:50.596476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f33019-429c-46a0-9bc2-83be8e754bf7 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:50.625056 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-27a2d7fe-e92a-4f9b-9cef-45f5c4706946 None None] MFA Rules not processed for user `ed33ed788b5d47ffa82ff121f7935cb2`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:50.627289 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-27a2d7fe-e92a-4f9b-9cef-45f5c4706946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:50.690366 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68be1f77-b5b2-4d50-8540-969b2b352bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:50.691460 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be1f77-b5b2-4d50-8540-969b2b352bf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:50.691983 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be1f77-b5b2-4d50-8540-969b2b352bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:50.692604 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68be1f77-b5b2-4d50-8540-969b2b352bf7 admin admin] PATH_INFO: `/v3/users/600cc6667bcb48539029bc494627728a` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:50.694690 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68be1f77-b5b2-4d50-8540-969b2b352bf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=600cc6667bcb48539029bc494627728a)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:50.698304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68be1f77-b5b2-4d50-8540-969b2b352bf7 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:50.723966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 968/1950] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:50.729680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2d7fe-e92a-4f9b-9cef-45f5c4706946 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:50.744526 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2d7fe-e92a-4f9b-9cef-45f5c4706946 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:50.752532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a142f7e-a71d-4f3e-a1a4-6f65f857b6f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:50.753658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a142f7e-a71d-4f3e-a1a4-6f65f857b6f4 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:50.754039 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a142f7e-a71d-4f3e-a1a4-6f65f857b6f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:50.758731 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a142f7e-a71d-4f3e-a1a4-6f65f857b6f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:50.758731 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a142f7e-a71d-4f3e-a1a4-6f65f857b6f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:50.758731 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a142f7e-a71d-4f3e-a1a4-6f65f857b6f4 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:50.889958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:50.901624 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 969/1951] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:55:50 2022] POST /identity/v3/projects => generated 380 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:55:50.902939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a142f7e-a71d-4f3e-a1a4-6f65f857b6f4 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:50.926185 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-68be1f77-b5b2-4d50-8540-969b2b352bf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '600cc6667bcb48539029bc494627728a'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:50.927473 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-68be1f77-b5b2-4d50-8540-969b2b352bf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '600cc6667bcb48539029bc494627728a'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:50.985136 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a142f7e-a71d-4f3e-a1a4-6f65f857b6f4 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:50.989920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ef184-b8b4-446a-9453-785de0c6e5e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:50.990548 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ef184-b8b4-446a-9453-785de0c6e5e6 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:50.990826 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ef184-b8b4-446a-9453-785de0c6e5e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:50.991134 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3ef184-b8b4-446a-9453-785de0c6e5e6 admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:50.991844 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ef184-b8b4-446a-9453-785de0c6e5e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:50.993802 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3ef184-b8b4-446a-9453-785de0c6e5e6 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:50.994375 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-68be1f77-b5b2-4d50-8540-969b2b352bf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '600cc6667bcb48539029bc494627728a'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:51.003795 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-68be1f77-b5b2-4d50-8540-969b2b352bf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '600cc6667bcb48539029bc494627728a'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:51.018116 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-68be1f77-b5b2-4d50-8540-969b2b352bf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '600cc6667bcb48539029bc494627728a'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:51.024506 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 983/1952] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:55:50 2022] DELETE /identity/v3/users/600cc6667bcb48539029bc494627728a => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:51.029279 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9903d5-bd5f-4750-b97b-2e9366a393c7 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:51.029572 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9903d5-bd5f-4750-b97b-2e9366a393c7 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:51.029850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9903d5-bd5f-4750-b97b-2e9366a393c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:51.051203 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-3f9903d5-bd5f-4750-b97b-2e9366a393c7 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:51.052434 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-3f9903d5-bd5f-4750-b97b-2e9366a393c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:51.131201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 984/1953] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:51 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:55:51.143669 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9903d5-bd5f-4750-b97b-2e9366a393c7 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:51.143846 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 970/1954] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:55:50 2022] POST /identity/v3/users => generated 291 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:55:51.145492 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-3f9903d5-bd5f-4750-b97b-2e9366a393c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:51.149185 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ef184-b8b4-446a-9453-785de0c6e5e6 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:51.152882 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-7d3ef184-b8b4-446a-9453-785de0c6e5e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:51.181885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3ef184-b8b4-446a-9453-785de0c6e5e6 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:51.190114 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ffc14-db8c-469d-96ec-82f6859e9ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:51.191123 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ffc14-db8c-469d-96ec-82f6859e9ada admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:51.191691 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ffc14-db8c-469d-96ec-82f6859e9ada admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:51.192164 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6ffc14-db8c-469d-96ec-82f6859e9ada admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:51.193130 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ffc14-db8c-469d-96ec-82f6859e9ada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:51.196156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6ffc14-db8c-469d-96ec-82f6859e9ada admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:51.214807 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 971/1955] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:51 2022] GET /identity/v3/roles => generated 1427 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:51.216583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ffc14-db8c-469d-96ec-82f6859e9ada admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:51.223910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9903d5-bd5f-4750-b97b-2e9366a393c7 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:51.227407 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6ffc14-db8c-469d-96ec-82f6859e9ada admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:51.228664 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8dd0c3d-bd46-4316-966c-7ce58ab8469c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:51.229319 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd0c3d-bd46-4316-966c-7ce58ab8469c service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:51.229605 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd0c3d-bd46-4316-966c-7ce58ab8469c service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:51.229991 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8dd0c3d-bd46-4316-966c-7ce58ab8469c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:51.250188 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f0db5d-c4b3-41d0-84fc-75b68b7936b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:51.250188 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0db5d-c4b3-41d0-84fc-75b68b7936b4 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:51.250188 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0db5d-c4b3-41d0-84fc-75b68b7936b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:51.250188 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0db5d-c4b3-41d0-84fc-75b68b7936b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:51.250589 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f0db5d-c4b3-41d0-84fc-75b68b7936b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:51.250589 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f0db5d-c4b3-41d0-84fc-75b68b7936b4 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:51.250755 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd0c3d-bd46-4316-966c-7ce58ab8469c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:51.250755 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8dd0c3d-bd46-4316-966c-7ce58ab8469c service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:51.257535 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 972/1956] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:51 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:51.259659 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f0db5d-c4b3-41d0-84fc-75b68b7936b4 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:51.265149 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 985/1957] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:51 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:55:51.272111 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f0db5d-c4b3-41d0-84fc-75b68b7936b4 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:51.279660 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb0cfb4-7a81-4627-b5dc-8546ffae861a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:51.280737 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb0cfb4-7a81-4627-b5dc-8546ffae861a admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:51.281281 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb0cfb4-7a81-4627-b5dc-8546ffae861a admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:51.281584 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb0cfb4-7a81-4627-b5dc-8546ffae861a admin admin] PATH_INFO: `/v3/projects/4071abc5bebe4e1fb0d83e05eaf717ce/users/a91266b70e104143af1bf1fd6551cbaa/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:51.300669 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb0cfb4-7a81-4627-b5dc-8546ffae861a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4071abc5bebe4e1fb0d83e05eaf717ce, user_id=a91266b70e104143af1bf1fd6551cbaa, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:51.304231 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb0cfb4-7a81-4627-b5dc-8546ffae861a admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:51.429436 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 973/1958] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:51 2022] PUT /identity/v3/projects/4071abc5bebe4e1fb0d83e05eaf717ce/users/a91266b70e104143af1bf1fd6551cbaa/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:55:51.431383 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704a974-a63c-4c7e-a0a8-b84a23e7c711 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:51.431780 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704a974-a63c-4c7e-a0a8-b84a23e7c711 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:51.432747 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e704a974-a63c-4c7e-a0a8-b84a23e7c711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:51.476403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-e704a974-a63c-4c7e-a0a8-b84a23e7c711 None None] MFA Rules not processed for user `a91266b70e104143af1bf1fd6551cbaa`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:51.476403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-e704a974-a63c-4c7e-a0a8-b84a23e7c711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:51.582836 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 986/1959] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:51 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:51.586934 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb0cfb4-7a81-4627-b5dc-8546ffae861a admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:51.685612 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb0cfb4-7a81-4627-b5dc-8546ffae861a admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:51.690993 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02ef52e-1d0c-4a31-b9d9-78d932e5afa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:51.692083 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ef52e-1d0c-4a31-b9d9-78d932e5afa5 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:51.692262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ef52e-1d0c-4a31-b9d9-78d932e5afa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:51.692677 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ef52e-1d0c-4a31-b9d9-78d932e5afa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:51.693545 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02ef52e-1d0c-4a31-b9d9-78d932e5afa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:51.695650 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02ef52e-1d0c-4a31-b9d9-78d932e5afa5 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:51.809755 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 974/1960] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:55:51 2022] POST /identity/v3/projects => generated 378 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:51.812455 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e704a974-a63c-4c7e-a0a8-b84a23e7c711 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:51.815544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-e704a974-a63c-4c7e-a0a8-b84a23e7c711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:51.827174 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02ef52e-1d0c-4a31-b9d9-78d932e5afa5 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:51.866402 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:51.871999 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e704a974-a63c-4c7e-a0a8-b84a23e7c711 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:51.991287 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585255c-5357-4b5c-9fec-40cba67b1c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:51.992211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585255c-5357-4b5c-9fec-40cba67b1c11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:51.992598 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585255c-5357-4b5c-9fec-40cba67b1c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:51.992980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585255c-5357-4b5c-9fec-40cba67b1c11 admin admin] PATH_INFO: `/v3/projects/f48684fc4aad4857b70e9a2a7d5582cd` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:51.994646 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9585255c-5357-4b5c-9fec-40cba67b1c11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f48684fc4aad4857b70e9a2a7d5582cd)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:51.997447 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9585255c-5357-4b5c-9fec-40cba67b1c11 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:52.025681 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02ef52e-1d0c-4a31-b9d9-78d932e5afa5 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:52.035327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c94146-ac07-4bb8-b5a3-8cdba83464f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:52.036133 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c94146-ac07-4bb8-b5a3-8cdba83464f3 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:52.036498 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c94146-ac07-4bb8-b5a3-8cdba83464f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:52.036857 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c94146-ac07-4bb8-b5a3-8cdba83464f3 admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:52.038161 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c94146-ac07-4bb8-b5a3-8cdba83464f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:52.040035 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c94146-ac07-4bb8-b5a3-8cdba83464f3 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:52.151499 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-9585255c-5357-4b5c-9fec-40cba67b1c11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f48684fc4aad4857b70e9a2a7d5582cd', 'request_id': 'req-9585255c-5357-4b5c-9fec-40cba67b1c11'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:52.158343 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-9585255c-5357-4b5c-9fec-40cba67b1c11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f48684fc4aad4857b70e9a2a7d5582cd', 'request_id': 'req-9585255c-5357-4b5c-9fec-40cba67b1c11'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:52.169660 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 975/1961] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:55:51 2022] POST /identity/v3/users => generated 296 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:52.175302 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c94146-ac07-4bb8-b5a3-8cdba83464f3 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:52.192825 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c94146-ac07-4bb8-b5a3-8cdba83464f3 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:52.253059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-9585255c-5357-4b5c-9fec-40cba67b1c11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f48684fc4aad4857b70e9a2a7d5582cd', 'request_id': 'req-9585255c-5357-4b5c-9fec-40cba67b1c11'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:52.264426 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6515b-ba3e-4991-98f8-b99efeecb9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:52.265065 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6515b-ba3e-4991-98f8-b99efeecb9e4 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:52.265320 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6515b-ba3e-4991-98f8-b99efeecb9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:52.265591 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc6515b-ba3e-4991-98f8-b99efeecb9e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:52.265949 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-9585255c-5357-4b5c-9fec-40cba67b1c11 admin admin] The token cache is being invalidate because project f48684fc4aad4857b70e9a2a7d5582cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:55:52.266246 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-9585255c-5357-4b5c-9fec-40cba67b1c11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:52.266439 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc6515b-ba3e-4991-98f8-b99efeecb9e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:52.266921 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-9585255c-5357-4b5c-9fec-40cba67b1c11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:52.267953 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc6515b-ba3e-4991-98f8-b99efeecb9e4 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:52.268509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 987/1962] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:55:51 2022] DELETE /identity/v3/projects/f48684fc4aad4857b70e9a2a7d5582cd => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:52.272472 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585255c-5357-4b5c-9fec-40cba67b1c11 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:52.275849 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-9585255c-5357-4b5c-9fec-40cba67b1c11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:52.283716 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 976/1963] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:52 2022] GET /identity/v3/roles => generated 1427 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:55:52.289146 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585255c-5357-4b5c-9fec-40cba67b1c11 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:52.291926 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6515b-ba3e-4991-98f8-b99efeecb9e4 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:52.294291 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-5fc6515b-ba3e-4991-98f8-b99efeecb9e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:52.295158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c32c2d-ce5f-43a0-aa1a-0d7c530eba15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:52.295864 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c32c2d-ce5f-43a0-aa1a-0d7c530eba15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:52.296179 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c32c2d-ce5f-43a0-aa1a-0d7c530eba15 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:52.296459 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c32c2d-ce5f-43a0-aa1a-0d7c530eba15 admin admin] PATH_INFO: `/v3/users/ce4d1a24e763414fb13a2a846e926ce1` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:52.297662 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c32c2d-ce5f-43a0-aa1a-0d7c530eba15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce4d1a24e763414fb13a2a846e926ce1)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:52.299667 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c32c2d-ce5f-43a0-aa1a-0d7c530eba15 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:52.308760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc6515b-ba3e-4991-98f8-b99efeecb9e4 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:52.313332 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff3e1ce-7dba-4ba6-b8f0-4044deff20e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:52.314090 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff3e1ce-7dba-4ba6-b8f0-4044deff20e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:52.314623 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff3e1ce-7dba-4ba6-b8f0-4044deff20e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:52.315034 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff3e1ce-7dba-4ba6-b8f0-4044deff20e8 admin admin] PATH_INFO: `/v3/projects/bb1ca0c45c604b5ea6c6e621d332534f/users/d7145d1324414a6099dc00fd4c15d130/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:52.343587 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff3e1ce-7dba-4ba6-b8f0-4044deff20e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb1ca0c45c604b5ea6c6e621d332534f, user_id=d7145d1324414a6099dc00fd4c15d130, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:52.349641 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff3e1ce-7dba-4ba6-b8f0-4044deff20e8 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:52.435663 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 977/1964] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:52 2022] PUT /identity/v3/projects/bb1ca0c45c604b5ea6c6e621d332534f/users/d7145d1324414a6099dc00fd4c15d130/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:55:52.441036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:52.443728 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff3e1ce-7dba-4ba6-b8f0-4044deff20e8 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:52.510282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff3e1ce-7dba-4ba6-b8f0-4044deff20e8 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:52.515961 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e738309-9b3a-4081-8720-05301e69c36e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:52.516573 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e738309-9b3a-4081-8720-05301e69c36e admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:52.516884 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e738309-9b3a-4081-8720-05301e69c36e admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:52.517154 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e738309-9b3a-4081-8720-05301e69c36e admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:52.517881 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e738309-9b3a-4081-8720-05301e69c36e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:52.519746 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e738309-9b3a-4081-8720-05301e69c36e admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:52.523632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-52c32c2d-ce5f-43a0-aa1a-0d7c530eba15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce4d1a24e763414fb13a2a846e926ce1'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:52.524288 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-52c32c2d-ce5f-43a0-aa1a-0d7c530eba15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce4d1a24e763414fb13a2a846e926ce1'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:52.530975 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 978/1965] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:52 2022] GET /identity/v3/roles => generated 1427 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:52.535399 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e738309-9b3a-4081-8720-05301e69c36e admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:52.546302 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e738309-9b3a-4081-8720-05301e69c36e admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:52.550978 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd422d6-ad29-4a2c-9cbc-6ebb1b4808f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:52.551669 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd422d6-ad29-4a2c-9cbc-6ebb1b4808f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:52.551950 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd422d6-ad29-4a2c-9cbc-6ebb1b4808f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:52.552224 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd422d6-ad29-4a2c-9cbc-6ebb1b4808f2 admin admin] PATH_INFO: `/v3/projects/bb1ca0c45c604b5ea6c6e621d332534f/users/d7145d1324414a6099dc00fd4c15d130/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:52.554203 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd422d6-ad29-4a2c-9cbc-6ebb1b4808f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb1ca0c45c604b5ea6c6e621d332534f, user_id=d7145d1324414a6099dc00fd4c15d130, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:52.556906 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd422d6-ad29-4a2c-9cbc-6ebb1b4808f2 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:52.571479 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 979/1966] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:52 2022] PUT /identity/v3/projects/bb1ca0c45c604b5ea6c6e621d332534f/users/d7145d1324414a6099dc00fd4c15d130/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:55:52.583209 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608c9c3-c677-4db9-83fa-a45a729b170d None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:52.583495 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608c9c3-c677-4db9-83fa-a45a729b170d None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:52.583775 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2608c9c3-c677-4db9-83fa-a45a729b170d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:52.595402 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-52c32c2d-ce5f-43a0-aa1a-0d7c530eba15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce4d1a24e763414fb13a2a846e926ce1'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:52.606985 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-52c32c2d-ce5f-43a0-aa1a-0d7c530eba15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce4d1a24e763414fb13a2a846e926ce1'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:52.622246 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-52c32c2d-ce5f-43a0-aa1a-0d7c530eba15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce4d1a24e763414fb13a2a846e926ce1'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:52.623521 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-2608c9c3-c677-4db9-83fa-a45a729b170d None None] MFA Rules not processed for user `d7145d1324414a6099dc00fd4c15d130`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:52.624859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-2608c9c3-c677-4db9-83fa-a45a729b170d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:52.642690 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 988/1967] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:55:52 2022] DELETE /identity/v3/users/ce4d1a24e763414fb13a2a846e926ce1 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:55:52.693565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:52.702809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 980/1968] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:52 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:55:52.714763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c32c2d-ce5f-43a0-aa1a-0d7c530eba15 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:52.717412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-52c32c2d-ce5f-43a0-aa1a-0d7c530eba15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:52.775992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c32c2d-ce5f-43a0-aa1a-0d7c530eba15 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:52.780634 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74742367-d778-48aa-9468-203bf6978434 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:52.781259 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74742367-d778-48aa-9468-203bf6978434 service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:52.781540 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74742367-d778-48aa-9468-203bf6978434 service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:52.781812 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74742367-d778-48aa-9468-203bf6978434 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:52.836256 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:52.850072 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74742367-d778-48aa-9468-203bf6978434 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:52.851512 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74742367-d778-48aa-9468-203bf6978434 service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:52.861870 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 989/1969] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:52 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:52.875915 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:52.977372 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2608c9c3-c677-4db9-83fa-a45a729b170d None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:52.978778 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-2608c9c3-c677-4db9-83fa-a45a729b170d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:53.036096 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2608c9c3-c677-4db9-83fa-a45a729b170d None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:53.039306 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74742367-d778-48aa-9468-203bf6978434 service cinder] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:53.040365 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db302b0-fd94-4b00-9c86-bead081290bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:53.040992 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db302b0-fd94-4b00-9c86-bead081290bc service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:53.041274 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db302b0-fd94-4b00-9c86-bead081290bc service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:53.041547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db302b0-fd94-4b00-9c86-bead081290bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:53.044164 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-4db302b0-fd94-4b00-9c86-bead081290bc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:53.044457 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-74742367-d778-48aa-9468-203bf6978434 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:53.099083 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:53.125005 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:53.149603 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db302b0-fd94-4b00-9c86-bead081290bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:53.153016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db302b0-fd94-4b00-9c86-bead081290bc service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:53.162050 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74742367-d778-48aa-9468-203bf6978434 service cinder] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:53.166995 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2022919-c324-4156-bef7-44dfd41103ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:53.167663 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2022919-c324-4156-bef7-44dfd41103ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:53.168097 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2022919-c324-4156-bef7-44dfd41103ef admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:53.168949 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2022919-c324-4156-bef7-44dfd41103ef admin admin] PATH_INFO: `/v3/projects/37f86be5f17f4a3dba854033f38f5745` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:53.170388 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2022919-c324-4156-bef7-44dfd41103ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=37f86be5f17f4a3dba854033f38f5745)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:53.172412 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2022919-c324-4156-bef7-44dfd41103ef admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:53.173099 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 981/1970] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:52 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:53.263616 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c2022919-c324-4156-bef7-44dfd41103ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37f86be5f17f4a3dba854033f38f5745', 'request_id': 'req-c2022919-c324-4156-bef7-44dfd41103ef'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:53.269736 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c2022919-c324-4156-bef7-44dfd41103ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37f86be5f17f4a3dba854033f38f5745', 'request_id': 'req-c2022919-c324-4156-bef7-44dfd41103ef'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:53.309519 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:53.326368 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c2022919-c324-4156-bef7-44dfd41103ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37f86be5f17f4a3dba854033f38f5745', 'request_id': 'req-c2022919-c324-4156-bef7-44dfd41103ef'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:53.339371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c2022919-c324-4156-bef7-44dfd41103ef admin admin] The token cache is being invalidate because project 37f86be5f17f4a3dba854033f38f5745 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:55:53.341047 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c2022919-c324-4156-bef7-44dfd41103ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:53.343370 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:53.344230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c2022919-c324-4156-bef7-44dfd41103ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:53.346461 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 990/1971] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:55:53 2022] DELETE /identity/v3/projects/37f86be5f17f4a3dba854033f38f5745 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:53.351305 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0d95b-5f48-4cb4-9cd7-39492fd25e3f None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:53.351592 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0d95b-5f48-4cb4-9cd7-39492fd25e3f None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:53.351859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0d95b-5f48-4cb4-9cd7-39492fd25e3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:53.379162 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-8cc0d95b-5f48-4cb4-9cd7-39492fd25e3f None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:53.380611 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-8cc0d95b-5f48-4cb4-9cd7-39492fd25e3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:53.435894 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:53.475772 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 982/1972] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:53 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:53.484315 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e572a07-1d5c-456a-9a0a-aa0986094dc1 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:53.484605 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e572a07-1d5c-456a-9a0a-aa0986094dc1 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:53.485134 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e572a07-1d5c-456a-9a0a-aa0986094dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:53.505974 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-9e572a07-1d5c-456a-9a0a-aa0986094dc1 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:53.507245 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-9e572a07-1d5c-456a-9a0a-aa0986094dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:53.517082 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 991/1973] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:53 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:53.523146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0d95b-5f48-4cb4-9cd7-39492fd25e3f None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:53.539338 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0d95b-5f48-4cb4-9cd7-39492fd25e3f None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:53.546017 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2210ee-9b32-495f-835d-8a00ae50d416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:53.546198 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2210ee-9b32-495f-835d-8a00ae50d416 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:53.546834 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2210ee-9b32-495f-835d-8a00ae50d416 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:53.546834 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2210ee-9b32-495f-835d-8a00ae50d416 admin admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:53.547515 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2210ee-9b32-495f-835d-8a00ae50d416 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:55:53.547818 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2210ee-9b32-495f-835d-8a00ae50d416 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:53.549479 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2210ee-9b32-495f-835d-8a00ae50d416 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:53.562374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 983/1974] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:55:53 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:55:53.572086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e572a07-1d5c-456a-9a0a-aa0986094dc1 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:53.585527 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e572a07-1d5c-456a-9a0a-aa0986094dc1 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:53.591468 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb918f-121b-4a1b-a9c9-1c7262666a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:53.592726 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb918f-121b-4a1b-a9c9-1c7262666a5a admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:53.593018 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb918f-121b-4a1b-a9c9-1c7262666a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:53.593327 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abb918f-121b-4a1b-a9c9-1c7262666a5a admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:53.594034 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abb918f-121b-4a1b-a9c9-1c7262666a5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:53.596019 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abb918f-121b-4a1b-a9c9-1c7262666a5a admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:53.724563 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 992/1975] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:55:53 2022] POST /identity/v3/projects => generated 384 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:53.728578 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2210ee-9b32-495f-835d-8a00ae50d416 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:53.735424 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:53.745037 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2210ee-9b32-495f-835d-8a00ae50d416 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:53.808319 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:53.830075 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617efda7-f175-4a96-afcb-8a9b841261d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:53.831219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617efda7-f175-4a96-afcb-8a9b841261d8 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:53.831342 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617efda7-f175-4a96-afcb-8a9b841261d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:53.831843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617efda7-f175-4a96-afcb-8a9b841261d8 admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:53.832715 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617efda7-f175-4a96-afcb-8a9b841261d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:53.835192 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617efda7-f175-4a96-afcb-8a9b841261d8 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:53.940010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 984/1976] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:55:53 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:53.944407 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb918f-121b-4a1b-a9c9-1c7262666a5a admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:53.956489 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abb918f-121b-4a1b-a9c9-1c7262666a5a admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:53.963564 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af42c9-53f3-46d4-aa94-3cf2b7e0b451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:53.964496 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af42c9-53f3-46d4-aa94-3cf2b7e0b451 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:53.965138 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af42c9-53f3-46d4-aa94-3cf2b7e0b451 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:53.967471 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12af42c9-53f3-46d4-aa94-3cf2b7e0b451 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:53.968243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af42c9-53f3-46d4-aa94-3cf2b7e0b451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:53.969943 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12af42c9-53f3-46d4-aa94-3cf2b7e0b451 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:53.981832 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 993/1977] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:53 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:53.986136 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617efda7-f175-4a96-afcb-8a9b841261d8 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:54.003621 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617efda7-f175-4a96-afcb-8a9b841261d8 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:54.019062 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75f5f90-0a08-4905-91b6-8641dcfd2cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:54.019757 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f5f90-0a08-4905-91b6-8641dcfd2cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:54.020041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f5f90-0a08-4905-91b6-8641dcfd2cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:54.020324 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75f5f90-0a08-4905-91b6-8641dcfd2cf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:54.021057 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f5f90-0a08-4905-91b6-8641dcfd2cf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:54.022996 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75f5f90-0a08-4905-91b6-8641dcfd2cf0 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:54.047747 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 985/1978] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:53 2022] GET /identity/v3/roles => generated 1427 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:54.048964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af42c9-53f3-46d4-aa94-3cf2b7e0b451 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:54.066024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12af42c9-53f3-46d4-aa94-3cf2b7e0b451 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:54.067573 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7175a7-a6a3-4bc2-8892-0ccc2834818f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:54.068357 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7175a7-a6a3-4bc2-8892-0ccc2834818f admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:54.068707 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7175a7-a6a3-4bc2-8892-0ccc2834818f admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:54.069084 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7175a7-a6a3-4bc2-8892-0ccc2834818f admin admin] PATH_INFO: `/v3/projects/05a777b267af438dbf034aee25765514/users/6dc3eeec3094439e805a3ea8e29f3e5c/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:54.086529 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7175a7-a6a3-4bc2-8892-0ccc2834818f admin admin] RBAC: Authorizing `identity:create_grant(project_id=05a777b267af438dbf034aee25765514, user_id=6dc3eeec3094439e805a3ea8e29f3e5c, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:54.089291 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7175a7-a6a3-4bc2-8892-0ccc2834818f admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:54.241116 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 994/1979] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:54 2022] PUT /identity/v3/projects/05a777b267af438dbf034aee25765514/users/6dc3eeec3094439e805a3ea8e29f3e5c/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:54.253915 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634aa473-b2c1-4eb8-b1b5-180737ab82d5 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:54.254198 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634aa473-b2c1-4eb8-b1b5-180737ab82d5 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:54.254476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634aa473-b2c1-4eb8-b1b5-180737ab82d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:54.291621 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-634aa473-b2c1-4eb8-b1b5-180737ab82d5 None None] MFA Rules not processed for user `6dc3eeec3094439e805a3ea8e29f3e5c`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:54.293690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-634aa473-b2c1-4eb8-b1b5-180737ab82d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:54.364846 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 986/1980] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:54 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:54.369418 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7175a7-a6a3-4bc2-8892-0ccc2834818f admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:54.440073 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7175a7-a6a3-4bc2-8892-0ccc2834818f admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:54.445256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6876ac8-77b8-4c9b-8b26-490f3a3e8922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:54.446007 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6876ac8-77b8-4c9b-8b26-490f3a3e8922 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:54.446285 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6876ac8-77b8-4c9b-8b26-490f3a3e8922 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:54.448131 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6876ac8-77b8-4c9b-8b26-490f3a3e8922 admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:54.448131 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6876ac8-77b8-4c9b-8b26-490f3a3e8922 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:54.449278 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6876ac8-77b8-4c9b-8b26-490f3a3e8922 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:54.503542 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:54.530690 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 995/1981] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:55:54 2022] POST /identity/v3/projects => generated 382 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:54.532764 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:54.534612 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634aa473-b2c1-4eb8-b1b5-180737ab82d5 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:54.546784 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634aa473-b2c1-4eb8-b1b5-180737ab82d5 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:54.610308 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08faa7-93c9-4f14-a82b-f5b95b52a83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:54.610931 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08faa7-93c9-4f14-a82b-f5b95b52a83e admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:54.611240 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08faa7-93c9-4f14-a82b-f5b95b52a83e admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:54.611638 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba08faa7-93c9-4f14-a82b-f5b95b52a83e admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:54.612194 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba08faa7-93c9-4f14-a82b-f5b95b52a83e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:54.614123 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba08faa7-93c9-4f14-a82b-f5b95b52a83e admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:54.694074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 987/1982] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:55:54 2022] POST /identity/v3/users => generated 298 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:55:54.698254 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6876ac8-77b8-4c9b-8b26-490f3a3e8922 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:54.709520 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6876ac8-77b8-4c9b-8b26-490f3a3e8922 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:54.722893 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523cac7-474e-4287-8a12-b262461b29a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:54.723638 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523cac7-474e-4287-8a12-b262461b29a6 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:54.723855 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523cac7-474e-4287-8a12-b262461b29a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:54.724136 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e523cac7-474e-4287-8a12-b262461b29a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:54.726376 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e523cac7-474e-4287-8a12-b262461b29a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:54.727576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e523cac7-474e-4287-8a12-b262461b29a6 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:54.740396 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 996/1983] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:54 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:54.744695 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08faa7-93c9-4f14-a82b-f5b95b52a83e admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:54.779790 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba08faa7-93c9-4f14-a82b-f5b95b52a83e admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:54.785636 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff87355-0c1c-4a5c-9a82-58d06b8acb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:54.786619 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff87355-0c1c-4a5c-9a82-58d06b8acb6f admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:54.786911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff87355-0c1c-4a5c-9a82-58d06b8acb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:54.787228 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff87355-0c1c-4a5c-9a82-58d06b8acb6f admin admin] PATH_INFO: `/v3/projects/bc6fd15a8668400fb26e691916680960/users/cda9d2caddca41f5af2d72ba8796c828/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:54.810899 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff87355-0c1c-4a5c-9a82-58d06b8acb6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc6fd15a8668400fb26e691916680960, user_id=cda9d2caddca41f5af2d72ba8796c828, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:54.816915 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff87355-0c1c-4a5c-9a82-58d06b8acb6f admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:54.898778 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:54.915703 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 988/1984] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:54 2022] PUT /identity/v3/projects/bc6fd15a8668400fb26e691916680960/users/cda9d2caddca41f5af2d72ba8796c828/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:54.920818 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523cac7-474e-4287-8a12-b262461b29a6 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:54.930994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e523cac7-474e-4287-8a12-b262461b29a6 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:55.003463 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e53e3e-d71e-4e27-ab6c-0db0a6a5a1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:55.005107 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e53e3e-d71e-4e27-ab6c-0db0a6a5a1c6 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:55.005107 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e53e3e-d71e-4e27-ab6c-0db0a6a5a1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:55.005107 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e53e3e-d71e-4e27-ab6c-0db0a6a5a1c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:55.005516 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e53e3e-d71e-4e27-ab6c-0db0a6a5a1c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:55.006837 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e53e3e-d71e-4e27-ab6c-0db0a6a5a1c6 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:55.017030 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 997/1985] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:54 2022] GET /identity/v3/roles => generated 1427 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:55:55.021383 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff87355-0c1c-4a5c-9a82-58d06b8acb6f admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:55.031866 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff87355-0c1c-4a5c-9a82-58d06b8acb6f admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:55.038260 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:55.038381 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ae4da-2b8a-41e0-bccf-6bed03e56a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:55.038381 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ae4da-2b8a-41e0-bccf-6bed03e56a30 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:55.038381 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ae4da-2b8a-41e0-bccf-6bed03e56a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:55.038381 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ae4da-2b8a-41e0-bccf-6bed03e56a30 admin admin] PATH_INFO: `/v3/projects/bc6fd15a8668400fb26e691916680960/users/cda9d2caddca41f5af2d72ba8796c828/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:55.040005 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ae4da-2b8a-41e0-bccf-6bed03e56a30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc6fd15a8668400fb26e691916680960, user_id=cda9d2caddca41f5af2d72ba8796c828, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:55.042613 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ae4da-2b8a-41e0-bccf-6bed03e56a30 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:55.053825 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 989/1986] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:55 2022] PUT /identity/v3/projects/bc6fd15a8668400fb26e691916680960/users/cda9d2caddca41f5af2d72ba8796c828/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:55:55.065936 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997a594-3ceb-492c-a17e-4687b250c568 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:55.070503 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997a594-3ceb-492c-a17e-4687b250c568 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:55.070503 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a997a594-3ceb-492c-a17e-4687b250c568 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:55.099534 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-a997a594-3ceb-492c-a17e-4687b250c568 None None] MFA Rules not processed for user `cda9d2caddca41f5af2d72ba8796c828`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:55.099534 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-a997a594-3ceb-492c-a17e-4687b250c568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:55.181431 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 998/1987] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:55 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:55.193824 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ae4da-2b8a-41e0-bccf-6bed03e56a30 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:55.195584 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-ce6ae4da-2b8a-41e0-bccf-6bed03e56a30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:55.258896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ae4da-2b8a-41e0-bccf-6bed03e56a30 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:55.262557 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7c8af-d1b4-4b90-9fa1-cadc26ca6614 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:55.263210 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7c8af-d1b4-4b90-9fa1-cadc26ca6614 service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:55.263494 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7c8af-d1b4-4b90-9fa1-cadc26ca6614 service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:55.263769 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7c8af-d1b4-4b90-9fa1-cadc26ca6614 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:55.272619 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a7c8af-d1b4-4b90-9fa1-cadc26ca6614 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:55.274051 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a7c8af-d1b4-4b90-9fa1-cadc26ca6614 service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:55.284125 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 990/1988] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:55 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:55:55.424194 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a997a594-3ceb-492c-a17e-4687b250c568 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:55.434778 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a997a594-3ceb-492c-a17e-4687b250c568 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:55.435642 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:55.439573 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab450a7-4a1f-41b4-903d-2f02da893f52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:55.440260 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab450a7-4a1f-41b4-903d-2f02da893f52 service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:55.440548 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab450a7-4a1f-41b4-903d-2f02da893f52 service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:55.440846 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab450a7-4a1f-41b4-903d-2f02da893f52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:55.502269 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab450a7-4a1f-41b4-903d-2f02da893f52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:55.504851 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:55.506916 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab450a7-4a1f-41b4-903d-2f02da893f52 service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:55.517718 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 999/1989] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:55 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:55.869518 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:56.201526 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7c8af-d1b4-4b90-9fa1-cadc26ca6614 service cinder] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:56.214054 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:56.287930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7c8af-d1b4-4b90-9fa1-cadc26ca6614 service cinder] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:56.292971 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854f426-0e8e-457f-9f59-c6bb6c001f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:56.293774 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854f426-0e8e-457f-9f59-c6bb6c001f5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:56.294057 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854f426-0e8e-457f-9f59-c6bb6c001f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:56.294331 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2854f426-0e8e-457f-9f59-c6bb6c001f5d admin admin] PATH_INFO: `/v3/users/6dc3eeec3094439e805a3ea8e29f3e5c` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:56.295515 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2854f426-0e8e-457f-9f59-c6bb6c001f5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dc3eeec3094439e805a3ea8e29f3e5c)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:56.297415 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2854f426-0e8e-457f-9f59-c6bb6c001f5d admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:56.467392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2854f426-0e8e-457f-9f59-c6bb6c001f5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dc3eeec3094439e805a3ea8e29f3e5c'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:56.474774 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2854f426-0e8e-457f-9f59-c6bb6c001f5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dc3eeec3094439e805a3ea8e29f3e5c'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:56.475469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2854f426-0e8e-457f-9f59-c6bb6c001f5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dc3eeec3094439e805a3ea8e29f3e5c'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:56.538898 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2854f426-0e8e-457f-9f59-c6bb6c001f5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dc3eeec3094439e805a3ea8e29f3e5c'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:56.551466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2854f426-0e8e-457f-9f59-c6bb6c001f5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dc3eeec3094439e805a3ea8e29f3e5c'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:56.557457 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 991/1990] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:55:56 2022] DELETE /identity/v3/users/6dc3eeec3094439e805a3ea8e29f3e5c => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:55:56.563108 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa6340-d04c-494c-bd68-b7ebe8e0fcc7 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:56.563667 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa6340-d04c-494c-bd68-b7ebe8e0fcc7 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:56.564295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa6340-d04c-494c-bd68-b7ebe8e0fcc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:56.586816 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-32fa6340-d04c-494c-bd68-b7ebe8e0fcc7 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:56.588106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-32fa6340-d04c-494c-bd68-b7ebe8e0fcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:56.642606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:56.663646 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1000/1991] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:56 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:56.674227 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854f426-0e8e-457f-9f59-c6bb6c001f5d admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:56.675746 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-2854f426-0e8e-457f-9f59-c6bb6c001f5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:56.737359 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2854f426-0e8e-457f-9f59-c6bb6c001f5d admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:56.741078 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e81f6d-59b6-47b4-a871-3346e82f8658 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:56.741695 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e81f6d-59b6-47b4-a871-3346e82f8658 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:56.742031 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e81f6d-59b6-47b4-a871-3346e82f8658 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:56.752574 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e81f6d-59b6-47b4-a871-3346e82f8658 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:56.755156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e81f6d-59b6-47b4-a871-3346e82f8658 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:56.756693 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e81f6d-59b6-47b4-a871-3346e82f8658 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:56.769601 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 992/1992] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:56 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:57.164952 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa6340-d04c-494c-bd68-b7ebe8e0fcc7 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:57.166788 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-32fa6340-d04c-494c-bd68-b7ebe8e0fcc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:57.181966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa6340-d04c-494c-bd68-b7ebe8e0fcc7 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:57.190120 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab82c06-f837-4eb0-a41f-58c1ac802f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:57.191271 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab82c06-f837-4eb0-a41f-58c1ac802f79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:57.196289 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab82c06-f837-4eb0-a41f-58c1ac802f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:57.196289 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab82c06-f837-4eb0-a41f-58c1ac802f79 admin admin] PATH_INFO: `/v3/projects/05a777b267af438dbf034aee25765514` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:57.196289 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab82c06-f837-4eb0-a41f-58c1ac802f79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05a777b267af438dbf034aee25765514)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:57.198326 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab82c06-f837-4eb0-a41f-58c1ac802f79 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:57.318803 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-8ab82c06-f837-4eb0-a41f-58c1ac802f79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05a777b267af438dbf034aee25765514', 'request_id': 'req-8ab82c06-f837-4eb0-a41f-58c1ac802f79'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:57.326822 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-8ab82c06-f837-4eb0-a41f-58c1ac802f79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05a777b267af438dbf034aee25765514', 'request_id': 'req-8ab82c06-f837-4eb0-a41f-58c1ac802f79'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:57.394619 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-8ab82c06-f837-4eb0-a41f-58c1ac802f79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05a777b267af438dbf034aee25765514', 'request_id': 'req-8ab82c06-f837-4eb0-a41f-58c1ac802f79'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:57.410000 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-8ab82c06-f837-4eb0-a41f-58c1ac802f79 admin admin] The token cache is being invalidate because project 05a777b267af438dbf034aee25765514 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:55:57.410454 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-8ab82c06-f837-4eb0-a41f-58c1ac802f79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:57.411669 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-8ab82c06-f837-4eb0-a41f-58c1ac802f79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:57.414806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1001/1993] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:55:57 2022] DELETE /identity/v3/projects/05a777b267af438dbf034aee25765514 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:57.419746 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e81f6d-59b6-47b4-a871-3346e82f8658 service neutron] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:57.432918 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e81f6d-59b6-47b4-a871-3346e82f8658 service neutron] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:57.450246 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab82c06-f837-4eb0-a41f-58c1ac802f79 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:57.451745 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-8ab82c06-f837-4eb0-a41f-58c1ac802f79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:57.496522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:57.542601 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6392be90-8901-42f3-8642-9aeeb5f4cf63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:57.544283 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392be90-8901-42f3-8642-9aeeb5f4cf63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:57.545121 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392be90-8901-42f3-8642-9aeeb5f4cf63 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:57.545538 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392be90-8901-42f3-8642-9aeeb5f4cf63 admin admin] PATH_INFO: `/v3/users/cda9d2caddca41f5af2d72ba8796c828` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:57.546854 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6392be90-8901-42f3-8642-9aeeb5f4cf63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cda9d2caddca41f5af2d72ba8796c828)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:57.549121 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6392be90-8901-42f3-8642-9aeeb5f4cf63 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:57.553274 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab82c06-f837-4eb0-a41f-58c1ac802f79 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:57.558901 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dea7e9-575d-4d0c-8b8d-87465dd378eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:57.559667 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dea7e9-575d-4d0c-8b8d-87465dd378eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:57.559952 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dea7e9-575d-4d0c-8b8d-87465dd378eb admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:57.560235 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dea7e9-575d-4d0c-8b8d-87465dd378eb admin admin] PATH_INFO: `/v3/users/ed33ed788b5d47ffa82ff121f7935cb2` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:57.561445 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dea7e9-575d-4d0c-8b8d-87465dd378eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed33ed788b5d47ffa82ff121f7935cb2)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:57.563417 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dea7e9-575d-4d0c-8b8d-87465dd378eb admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:57.788152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-6392be90-8901-42f3-8642-9aeeb5f4cf63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cda9d2caddca41f5af2d72ba8796c828'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:57.788801 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-b6dea7e9-575d-4d0c-8b8d-87465dd378eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed33ed788b5d47ffa82ff121f7935cb2'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:57.789617 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-b6dea7e9-575d-4d0c-8b8d-87465dd378eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed33ed788b5d47ffa82ff121f7935cb2'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:57.796738 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-6392be90-8901-42f3-8642-9aeeb5f4cf63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cda9d2caddca41f5af2d72ba8796c828'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:57.797393 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-6392be90-8901-42f3-8642-9aeeb5f4cf63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cda9d2caddca41f5af2d72ba8796c828'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:57.879076 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-b6dea7e9-575d-4d0c-8b8d-87465dd378eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed33ed788b5d47ffa82ff121f7935cb2'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:57.894045 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-b6dea7e9-575d-4d0c-8b8d-87465dd378eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed33ed788b5d47ffa82ff121f7935cb2'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:57.900835 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:57.910335 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-b6dea7e9-575d-4d0c-8b8d-87465dd378eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed33ed788b5d47ffa82ff121f7935cb2'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:57.917534 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1002/1994] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:55:57 2022] DELETE /identity/v3/users/ed33ed788b5d47ffa82ff121f7935cb2 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:57.922229 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624844c-023b-4342-a3c5-dd5c1931145a None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:57.922519 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624844c-023b-4342-a3c5-dd5c1931145a None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:57.922826 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624844c-023b-4342-a3c5-dd5c1931145a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:57.941675 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-d624844c-023b-4342-a3c5-dd5c1931145a None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:57.942891 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-d624844c-023b-4342-a3c5-dd5c1931145a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:57.954261 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-6392be90-8901-42f3-8642-9aeeb5f4cf63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cda9d2caddca41f5af2d72ba8796c828'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:57.970511 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-6392be90-8901-42f3-8642-9aeeb5f4cf63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cda9d2caddca41f5af2d72ba8796c828'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:57.995463 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 993/1995] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:55:57 2022] DELETE /identity/v3/users/cda9d2caddca41f5af2d72ba8796c828 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:55:58.022953 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1003/1996] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:57 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:55:58.035189 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6392be90-8901-42f3-8642-9aeeb5f4cf63 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:58.038012 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-6392be90-8901-42f3-8642-9aeeb5f4cf63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:58.127382 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6392be90-8901-42f3-8642-9aeeb5f4cf63 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:58.132864 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55ddbb-4ed9-476a-ad74-a712a9069ba4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:58.133482 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55ddbb-4ed9-476a-ad74-a712a9069ba4 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:58.133762 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55ddbb-4ed9-476a-ad74-a712a9069ba4 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:58.134039 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a55ddbb-4ed9-476a-ad74-a712a9069ba4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:58.207613 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624844c-023b-4342-a3c5-dd5c1931145a None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:58.209221 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-d624844c-023b-4342-a3c5-dd5c1931145a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:58.209430 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a55ddbb-4ed9-476a-ad74-a712a9069ba4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:58.211313 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a55ddbb-4ed9-476a-ad74-a712a9069ba4 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:58.217071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:58.223325 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624844c-023b-4342-a3c5-dd5c1931145a None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:58.231871 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b3f548-50c2-428b-992d-a1bed5f1a6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:58.232537 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b3f548-50c2-428b-992d-a1bed5f1a6b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:58.232827 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b3f548-50c2-428b-992d-a1bed5f1a6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:58.233192 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b3f548-50c2-428b-992d-a1bed5f1a6b2 admin admin] PATH_INFO: `/v3/users/c584d40a93104884b26a2b4b73bdcd2b` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:58.234432 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b3f548-50c2-428b-992d-a1bed5f1a6b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c584d40a93104884b26a2b4b73bdcd2b)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:58.236535 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b3f548-50c2-428b-992d-a1bed5f1a6b2 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:58.243470 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 994/1997] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:58 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:55:58.437215 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-35b3f548-50c2-428b-992d-a1bed5f1a6b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c584d40a93104884b26a2b4b73bdcd2b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:58.437896 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-35b3f548-50c2-428b-992d-a1bed5f1a6b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c584d40a93104884b26a2b4b73bdcd2b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:58.469034 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55ddbb-4ed9-476a-ad74-a712a9069ba4 service neutron] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:58.470868 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-5a55ddbb-4ed9-476a-ad74-a712a9069ba4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:58.489744 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a55ddbb-4ed9-476a-ad74-a712a9069ba4 service neutron] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:58.493968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a7f0a-9975-4431-bb67-675e67780b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:58.503965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a7f0a-9975-4431-bb67-675e67780b5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:58.507547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a7f0a-9975-4431-bb67-675e67780b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:58.508209 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7a7f0a-9975-4431-bb67-675e67780b5c admin admin] PATH_INFO: `/v3/projects/bc6fd15a8668400fb26e691916680960` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:58.510078 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7a7f0a-9975-4431-bb67-675e67780b5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc6fd15a8668400fb26e691916680960)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:58.524498 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7a7f0a-9975-4431-bb67-675e67780b5c admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:58.543656 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-35b3f548-50c2-428b-992d-a1bed5f1a6b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c584d40a93104884b26a2b4b73bdcd2b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:58.553447 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-35b3f548-50c2-428b-992d-a1bed5f1a6b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c584d40a93104884b26a2b4b73bdcd2b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:58.575620 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-35b3f548-50c2-428b-992d-a1bed5f1a6b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c584d40a93104884b26a2b4b73bdcd2b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:58.583861 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1004/1998] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:55:58 2022] DELETE /identity/v3/users/c584d40a93104884b26a2b4b73bdcd2b => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:55:58.587809 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac4836-67e0-4e79-8fcd-94598ee29860 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:58.588188 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac4836-67e0-4e79-8fcd-94598ee29860 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:58.588470 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ac4836-67e0-4e79-8fcd-94598ee29860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:58.610692 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-06ac4836-67e0-4e79-8fcd-94598ee29860 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:58.615356 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-06ac4836-67e0-4e79-8fcd-94598ee29860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:58.660433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-de7a7f0a-9975-4431-bb67-675e67780b5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc6fd15a8668400fb26e691916680960', 'request_id': 'req-de7a7f0a-9975-4431-bb67-675e67780b5c'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:58.676771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:58.686430 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1005/1999] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:58 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:58.697169 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ac4836-67e0-4e79-8fcd-94598ee29860 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:58.698607 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-06ac4836-67e0-4e79-8fcd-94598ee29860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:58.756070 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-de7a7f0a-9975-4431-bb67-675e67780b5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc6fd15a8668400fb26e691916680960', 'request_id': 'req-de7a7f0a-9975-4431-bb67-675e67780b5c'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:58.761583 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ac4836-67e0-4e79-8fcd-94598ee29860 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:58.773341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-de7a7f0a-9975-4431-bb67-675e67780b5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc6fd15a8668400fb26e691916680960', 'request_id': 'req-de7a7f0a-9975-4431-bb67-675e67780b5c'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:58.785938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-de7a7f0a-9975-4431-bb67-675e67780b5c admin admin] The token cache is being invalidate because project bc6fd15a8668400fb26e691916680960 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:55:58.786371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-de7a7f0a-9975-4431-bb67-675e67780b5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:58.787072 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-de7a7f0a-9975-4431-bb67-675e67780b5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:58.788988 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 995/2000] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:55:58 2022] DELETE /identity/v3/projects/bc6fd15a8668400fb26e691916680960 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:58.793658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a7f0a-9975-4431-bb67-675e67780b5c admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:58.795222 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-de7a7f0a-9975-4431-bb67-675e67780b5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:58.816778 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d89df1-8be6-42a8-9d3e-e0834cdfb01a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:58.817317 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d89df1-8be6-42a8-9d3e-e0834cdfb01a service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:58.817600 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d89df1-8be6-42a8-9d3e-e0834cdfb01a service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:58.817876 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d89df1-8be6-42a8-9d3e-e0834cdfb01a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:58.819669 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-72d89df1-8be6-42a8-9d3e-e0834cdfb01a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:58.860053 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7a7f0a-9975-4431-bb67-675e67780b5c admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:58.865526 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c30a44-2438-4544-8884-9718b7cfc8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:58.866170 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c30a44-2438-4544-8884-9718b7cfc8fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:58.866476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c30a44-2438-4544-8884-9718b7cfc8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:58.866751 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c30a44-2438-4544-8884-9718b7cfc8fe admin admin] PATH_INFO: `/v3/projects/0ba41428b84447e99a7f0f7f143c7f90` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:58.868034 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c30a44-2438-4544-8884-9718b7cfc8fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ba41428b84447e99a7f0f7f143c7f90)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:58.869961 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c30a44-2438-4544-8884-9718b7cfc8fe admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:58.884559 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d89df1-8be6-42a8-9d3e-e0834cdfb01a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:58.886003 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d89df1-8be6-42a8-9d3e-e0834cdfb01a service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:58.896461 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1006/2001] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:58 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:55:58.904621 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8967f5-35ed-4e2c-969e-a137a7494a60 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:58.904914 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8967f5-35ed-4e2c-969e-a137a7494a60 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:58.905193 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8967f5-35ed-4e2c-969e-a137a7494a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:58.926863 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-1b8967f5-35ed-4e2c-969e-a137a7494a60 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:58.927647 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-1b8967f5-35ed-4e2c-969e-a137a7494a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:58.939261 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1007/2002] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:58 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:58.947462 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2f017-708f-4789-81e5-af87dabbe0cd None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:58.947753 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2f017-708f-4789-81e5-af87dabbe0cd None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:58.948032 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f2f017-708f-4789-81e5-af87dabbe0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:58.973073 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-57f2f017-708f-4789-81e5-af87dabbe0cd None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:58.974442 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-57f2f017-708f-4789-81e5-af87dabbe0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:58.993507 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-79c30a44-2438-4544-8884-9718b7cfc8fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ba41428b84447e99a7f0f7f143c7f90', 'request_id': 'req-79c30a44-2438-4544-8884-9718b7cfc8fe'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:59.039165 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1008/2003] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:58 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:59.043471 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2f017-708f-4789-81e5-af87dabbe0cd None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:59.053845 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f2f017-708f-4789-81e5-af87dabbe0cd None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:59.061336 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0699a9e3-ff97-4883-b62d-4af432fde410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:59.062080 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0699a9e3-ff97-4883-b62d-4af432fde410 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:59.062363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0699a9e3-ff97-4883-b62d-4af432fde410 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:59.062640 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0699a9e3-ff97-4883-b62d-4af432fde410 admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:59.063465 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0699a9e3-ff97-4883-b62d-4af432fde410 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:55:59.063805 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0699a9e3-ff97-4883-b62d-4af432fde410 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:59.065693 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0699a9e3-ff97-4883-b62d-4af432fde410 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:59.078699 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1009/2004] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:55:59 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:55:59.083041 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0699a9e3-ff97-4883-b62d-4af432fde410 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:59.107251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0699a9e3-ff97-4883-b62d-4af432fde410 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:59.107251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cacc48-b9ba-473e-a824-466c1213a1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:59.107251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cacc48-b9ba-473e-a824-466c1213a1fa admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:59.107251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cacc48-b9ba-473e-a824-466c1213a1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:59.107579 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cacc48-b9ba-473e-a824-466c1213a1fa admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:59.107579 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cacc48-b9ba-473e-a824-466c1213a1fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:59.107579 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cacc48-b9ba-473e-a824-466c1213a1fa admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:59.160898 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-79c30a44-2438-4544-8884-9718b7cfc8fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ba41428b84447e99a7f0f7f143c7f90', 'request_id': 'req-79c30a44-2438-4544-8884-9718b7cfc8fe'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:59.168485 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-79c30a44-2438-4544-8884-9718b7cfc8fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ba41428b84447e99a7f0f7f143c7f90', 'request_id': 'req-79c30a44-2438-4544-8884-9718b7cfc8fe'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:59.188395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-79c30a44-2438-4544-8884-9718b7cfc8fe admin admin] The token cache is being invalidate because project 0ba41428b84447e99a7f0f7f143c7f90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:55:59.188764 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-79c30a44-2438-4544-8884-9718b7cfc8fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:59.189432 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-79c30a44-2438-4544-8884-9718b7cfc8fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:59.191780 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1010/2005] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:55:59 2022] POST /identity/v3/projects => generated 372 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:55:59.197526 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 996/2006] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:55:58 2022] DELETE /identity/v3/projects/0ba41428b84447e99a7f0f7f143c7f90 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:59.198771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cacc48-b9ba-473e-a824-466c1213a1fa admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:59.203230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-b9cacc48-b9ba-473e-a824-466c1213a1fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:59.212194 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c30a44-2438-4544-8884-9718b7cfc8fe admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:59.213738 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-79c30a44-2438-4544-8884-9718b7cfc8fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:59.241145 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c30a44-2438-4544-8884-9718b7cfc8fe admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:59.312402 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cacc48-b9ba-473e-a824-466c1213a1fa admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:59.317616 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91fb42-94b5-4f9d-98ba-e3dbfa0bf121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:59.318486 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91fb42-94b5-4f9d-98ba-e3dbfa0bf121 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:59.319102 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91fb42-94b5-4f9d-98ba-e3dbfa0bf121 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:59.319426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91fb42-94b5-4f9d-98ba-e3dbfa0bf121 admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:59.320111 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f91fb42-94b5-4f9d-98ba-e3dbfa0bf121 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:59.322501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f91fb42-94b5-4f9d-98ba-e3dbfa0bf121 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:59.333257 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:59.347283 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7edbf-86df-4416-9e4b-4fb3a52bec45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:59.348204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7edbf-86df-4416-9e4b-4fb3a52bec45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:59.348571 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7edbf-86df-4416-9e4b-4fb3a52bec45 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:59.348908 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7edbf-86df-4416-9e4b-4fb3a52bec45 admin admin] PATH_INFO: `/v3/users/a91266b70e104143af1bf1fd6551cbaa` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:59.350161 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad7edbf-86df-4416-9e4b-4fb3a52bec45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a91266b70e104143af1bf1fd6551cbaa)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:59.352289 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad7edbf-86df-4416-9e4b-4fb3a52bec45 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:59.412122 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1011/2007] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:55:59 2022] POST /identity/v3/users => generated 287 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:55:59.416639 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91fb42-94b5-4f9d-98ba-e3dbfa0bf121 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:59.424685 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:59.428750 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91fb42-94b5-4f9d-98ba-e3dbfa0bf121 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:59.438805 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4356243c-f12e-4e41-901a-32e9721d9e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:59.439958 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356243c-f12e-4e41-901a-32e9721d9e13 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:59.439958 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356243c-f12e-4e41-901a-32e9721d9e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:59.440158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4356243c-f12e-4e41-901a-32e9721d9e13 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:59.440794 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4356243c-f12e-4e41-901a-32e9721d9e13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:59.443735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4356243c-f12e-4e41-901a-32e9721d9e13 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:59.459184 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1012/2008] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:59 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:55:59.459184 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4356243c-f12e-4e41-901a-32e9721d9e13 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:59.460605 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-4356243c-f12e-4e41-901a-32e9721d9e13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:59.474086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4356243c-f12e-4e41-901a-32e9721d9e13 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:59.478817 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1df66a-f0b5-4766-9b7e-95968a266e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:59.479492 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1df66a-f0b5-4766-9b7e-95968a266e0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:59.479776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1df66a-f0b5-4766-9b7e-95968a266e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:59.480057 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1df66a-f0b5-4766-9b7e-95968a266e0d admin admin] PATH_INFO: `/v3/projects/772ace22947e4a9893fd7c87ccb22905` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:59.481862 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1df66a-f0b5-4766-9b7e-95968a266e0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=772ace22947e4a9893fd7c87ccb22905)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:59.483822 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1df66a-f0b5-4766-9b7e-95968a266e0d admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:59.514471 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-1ad7edbf-86df-4416-9e4b-4fb3a52bec45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a91266b70e104143af1bf1fd6551cbaa'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:59.522047 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-1ad7edbf-86df-4416-9e4b-4fb3a52bec45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a91266b70e104143af1bf1fd6551cbaa'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:59.522763 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-1ad7edbf-86df-4416-9e4b-4fb3a52bec45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a91266b70e104143af1bf1fd6551cbaa'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:59.589140 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-1ad7edbf-86df-4416-9e4b-4fb3a52bec45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a91266b70e104143af1bf1fd6551cbaa'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:59.602288 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-1ad7edbf-86df-4416-9e4b-4fb3a52bec45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a91266b70e104143af1bf1fd6551cbaa'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:59.607978 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-cc1df66a-f0b5-4766-9b7e-95968a266e0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '772ace22947e4a9893fd7c87ccb22905', 'request_id': 'req-cc1df66a-f0b5-4766-9b7e-95968a266e0d'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:59.609717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 997/2009] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:55:59 2022] DELETE /identity/v3/users/a91266b70e104143af1bf1fd6551cbaa => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:59.613444 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7edbf-86df-4416-9e4b-4fb3a52bec45 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:59.614872 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-cc1df66a-f0b5-4766-9b7e-95968a266e0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '772ace22947e4a9893fd7c87ccb22905', 'request_id': 'req-cc1df66a-f0b5-4766-9b7e-95968a266e0d'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:59.614988 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-1ad7edbf-86df-4416-9e4b-4fb3a52bec45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:59.686385 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7edbf-86df-4416-9e4b-4fb3a52bec45 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:59.691511 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d385d-92c7-4207-a8b7-d8647fcefd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:59.692132 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d385d-92c7-4207-a8b7-d8647fcefd5d admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:59.692408 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d385d-92c7-4207-a8b7-d8647fcefd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:59.692687 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604d385d-92c7-4207-a8b7-d8647fcefd5d admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:59.693368 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604d385d-92c7-4207-a8b7-d8647fcefd5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:59.694960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604d385d-92c7-4207-a8b7-d8647fcefd5d admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:59.703715 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-cc1df66a-f0b5-4766-9b7e-95968a266e0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '772ace22947e4a9893fd7c87ccb22905', 'request_id': 'req-cc1df66a-f0b5-4766-9b7e-95968a266e0d'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:59.706261 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 998/2010] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:55:59 2022] GET /identity/v3/roles => generated 1427 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:55:59.710708 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d385d-92c7-4207-a8b7-d8647fcefd5d admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:59.717606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-cc1df66a-f0b5-4766-9b7e-95968a266e0d admin admin] The token cache is being invalidate because project 772ace22947e4a9893fd7c87ccb22905 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:55:59.717906 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-cc1df66a-f0b5-4766-9b7e-95968a266e0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:59.718675 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-cc1df66a-f0b5-4766-9b7e-95968a266e0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:55:59.720321 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1013/2011] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:55:59 2022] DELETE /identity/v3/projects/772ace22947e4a9893fd7c87ccb22905 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:59.722244 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604d385d-92c7-4207-a8b7-d8647fcefd5d admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:59.724206 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1df66a-f0b5-4766-9b7e-95968a266e0d admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:59.725681 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-cc1df66a-f0b5-4766-9b7e-95968a266e0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:59.728564 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8b4c8a-55d7-4bb1-bdb7-aa7204eab501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:59.729334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b4c8a-55d7-4bb1-bdb7-aa7204eab501 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:59.729668 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b4c8a-55d7-4bb1-bdb7-aa7204eab501 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:59.730085 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8b4c8a-55d7-4bb1-bdb7-aa7204eab501 admin admin] PATH_INFO: `/v3/projects/b7965b49eb5547cc8de3f28b8c53f341/users/bec9ea3f61b7483da3a629b290000fec/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:59.747465 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8b4c8a-55d7-4bb1-bdb7-aa7204eab501 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7965b49eb5547cc8de3f28b8c53f341, user_id=bec9ea3f61b7483da3a629b290000fec, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:59.750115 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8b4c8a-55d7-4bb1-bdb7-aa7204eab501 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:59.771149 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:55:59.830920 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1df66a-f0b5-4766-9b7e-95968a266e0d admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:55:59.836877 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 999/2012] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:55:59 2022] PUT /identity/v3/projects/b7965b49eb5547cc8de3f28b8c53f341/users/bec9ea3f61b7483da3a629b290000fec/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:55:59.844027 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903b664-6264-43da-a236-4334d3e82453 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:59.845354 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903b664-6264-43da-a236-4334d3e82453 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:59.846038 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f903b664-6264-43da-a236-4334d3e82453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:59.877112 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-f903b664-6264-43da-a236-4334d3e82453 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:55:59.878924 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f903b664-6264-43da-a236-4334d3e82453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:55:59.904392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707cb4d-17da-41e8-9c56-78b76c7edecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:55:59.905117 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707cb4d-17da-41e8-9c56-78b76c7edecb admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:55:59.905402 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707cb4d-17da-41e8-9c56-78b76c7edecb admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:55:59.905682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2707cb4d-17da-41e8-9c56-78b76c7edecb admin admin] PATH_INFO: `/v3/users/40546cb4010f4163b5c3374b09e55996` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:55:59.907159 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707cb4d-17da-41e8-9c56-78b76c7edecb admin admin] RBAC: Authorizing `identity:delete_user(user_id=40546cb4010f4163b5c3374b09e55996)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:55:59.909661 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2707cb4d-17da-41e8-9c56-78b76c7edecb admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:55:59.972572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1000/2013] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:55:59 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:55:59.982726 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903b664-6264-43da-a236-4334d3e82453 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:55:59.984128 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f903b664-6264-43da-a236-4334d3e82453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:00.047348 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f903b664-6264-43da-a236-4334d3e82453 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:00.050828 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2fbc2-f732-4e91-a71d-4bebdd02e4b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:00.050828 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2fbc2-f732-4e91-a71d-4bebdd02e4b8 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:00.050828 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2fbc2-f732-4e91-a71d-4bebdd02e4b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:00.050828 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2fbc2-f732-4e91-a71d-4bebdd02e4b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:00.060529 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db2fbc2-f732-4e91-a71d-4bebdd02e4b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:00.061964 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db2fbc2-f732-4e91-a71d-4bebdd02e4b8 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:00.072001 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1001/2014] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:55:59 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:56:00.073012 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2fbc2-f732-4e91-a71d-4bebdd02e4b8 service neutron] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:00.075103 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-6db2fbc2-f732-4e91-a71d-4bebdd02e4b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:00.089440 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2fbc2-f732-4e91-a71d-4bebdd02e4b8 service neutron] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:00.095098 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1adc6-ffb0-4c8a-9818-1284981889a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:00.095813 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1adc6-ffb0-4c8a-9818-1284981889a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:00.102639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1adc6-ffb0-4c8a-9818-1284981889a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:00.102639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1adc6-ffb0-4c8a-9818-1284981889a5 admin admin] PATH_INFO: `/v3/projects/4071abc5bebe4e1fb0d83e05eaf717ce` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:00.102639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf1adc6-ffb0-4c8a-9818-1284981889a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4071abc5bebe4e1fb0d83e05eaf717ce)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:00.102639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf1adc6-ffb0-4c8a-9818-1284981889a5 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:00.143546 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:00.157879 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-2707cb4d-17da-41e8-9c56-78b76c7edecb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40546cb4010f4163b5c3374b09e55996'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:00.158537 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-2707cb4d-17da-41e8-9c56-78b76c7edecb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40546cb4010f4163b5c3374b09e55996'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:00.235099 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-2707cb4d-17da-41e8-9c56-78b76c7edecb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40546cb4010f4163b5c3374b09e55996'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:00.235588 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-8bf1adc6-ffb0-4c8a-9818-1284981889a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4071abc5bebe4e1fb0d83e05eaf717ce', 'request_id': 'req-8bf1adc6-ffb0-4c8a-9818-1284981889a5'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:00.244196 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-2707cb4d-17da-41e8-9c56-78b76c7edecb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40546cb4010f4163b5c3374b09e55996'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:00.257534 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-2707cb4d-17da-41e8-9c56-78b76c7edecb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40546cb4010f4163b5c3374b09e55996'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:00.264549 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1014/2015] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:55:59 2022] DELETE /identity/v3/users/40546cb4010f4163b5c3374b09e55996 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:56:00.310320 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-8bf1adc6-ffb0-4c8a-9818-1284981889a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4071abc5bebe4e1fb0d83e05eaf717ce', 'request_id': 'req-8bf1adc6-ffb0-4c8a-9818-1284981889a5'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:00.317124 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-8bf1adc6-ffb0-4c8a-9818-1284981889a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4071abc5bebe4e1fb0d83e05eaf717ce', 'request_id': 'req-8bf1adc6-ffb0-4c8a-9818-1284981889a5'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:00.330007 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-8bf1adc6-ffb0-4c8a-9818-1284981889a5 admin admin] The token cache is being invalidate because project 4071abc5bebe4e1fb0d83e05eaf717ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:56:00.330533 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-8bf1adc6-ffb0-4c8a-9818-1284981889a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:00.332404 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-8bf1adc6-ffb0-4c8a-9818-1284981889a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:00.333731 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1002/2016] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:56:00 2022] DELETE /identity/v3/projects/4071abc5bebe4e1fb0d83e05eaf717ce => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:00.337785 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707cb4d-17da-41e8-9c56-78b76c7edecb admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:00.339278 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-2707cb4d-17da-41e8-9c56-78b76c7edecb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:00.409573 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2707cb4d-17da-41e8-9c56-78b76c7edecb admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:00.414799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70678afb-5308-4671-b78c-b81dd4f16457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:00.415484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70678afb-5308-4671-b78c-b81dd4f16457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:00.415771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70678afb-5308-4671-b78c-b81dd4f16457 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:00.416051 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70678afb-5308-4671-b78c-b81dd4f16457 admin admin] PATH_INFO: `/v3/users/d7145d1324414a6099dc00fd4c15d130` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:00.417316 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70678afb-5308-4671-b78c-b81dd4f16457 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7145d1324414a6099dc00fd4c15d130)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:00.420481 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70678afb-5308-4671-b78c-b81dd4f16457 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:00.543934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:00.653497 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-70678afb-5308-4671-b78c-b81dd4f16457 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7145d1324414a6099dc00fd4c15d130'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:00.654568 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-70678afb-5308-4671-b78c-b81dd4f16457 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7145d1324414a6099dc00fd4c15d130'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:00.841360 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-70678afb-5308-4671-b78c-b81dd4f16457 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7145d1324414a6099dc00fd4c15d130'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:00.849203 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-70678afb-5308-4671-b78c-b81dd4f16457 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7145d1324414a6099dc00fd4c15d130'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:00.862032 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-70678afb-5308-4671-b78c-b81dd4f16457 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7145d1324414a6099dc00fd4c15d130'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:00.868534 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1015/2017] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:56:00 2022] DELETE /identity/v3/users/d7145d1324414a6099dc00fd4c15d130 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:01.081521 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1adc6-ffb0-4c8a-9818-1284981889a5 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:01.083074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-8bf1adc6-ffb0-4c8a-9818-1284981889a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:01.126557 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:01.170926 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1adc6-ffb0-4c8a-9818-1284981889a5 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:01.242998 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e78acaa-44b3-4311-bf8b-4fe050c9ba0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:01.244827 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78acaa-44b3-4311-bf8b-4fe050c9ba0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:01.245117 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78acaa-44b3-4311-bf8b-4fe050c9ba0f admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:01.245398 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e78acaa-44b3-4311-bf8b-4fe050c9ba0f admin admin] PATH_INFO: `/v3/projects/6da1fab8502c40b4b5edccb5e97e77c2` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:01.247523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78acaa-44b3-4311-bf8b-4fe050c9ba0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6da1fab8502c40b4b5edccb5e97e77c2)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:01.249697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e78acaa-44b3-4311-bf8b-4fe050c9ba0f admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:01.362823 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70678afb-5308-4671-b78c-b81dd4f16457 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:01.364444 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-70678afb-5308-4671-b78c-b81dd4f16457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:01.401868 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-6e78acaa-44b3-4311-bf8b-4fe050c9ba0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6da1fab8502c40b4b5edccb5e97e77c2', 'request_id': 'req-6e78acaa-44b3-4311-bf8b-4fe050c9ba0f'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:01.403344 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70678afb-5308-4671-b78c-b81dd4f16457 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:01.463069 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-6e78acaa-44b3-4311-bf8b-4fe050c9ba0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6da1fab8502c40b4b5edccb5e97e77c2', 'request_id': 'req-6e78acaa-44b3-4311-bf8b-4fe050c9ba0f'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:01.468133 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e755801c-36bd-4dee-8bd7-84715c2fd477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:01.468979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755801c-36bd-4dee-8bd7-84715c2fd477 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:01.469366 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755801c-36bd-4dee-8bd7-84715c2fd477 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:01.469562 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755801c-36bd-4dee-8bd7-84715c2fd477 admin admin] PATH_INFO: `/v3/projects/bb1ca0c45c604b5ea6c6e621d332534f` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:01.470676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-6e78acaa-44b3-4311-bf8b-4fe050c9ba0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6da1fab8502c40b4b5edccb5e97e77c2', 'request_id': 'req-6e78acaa-44b3-4311-bf8b-4fe050c9ba0f'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:01.470954 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e755801c-36bd-4dee-8bd7-84715c2fd477 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb1ca0c45c604b5ea6c6e621d332534f)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:01.473499 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e755801c-36bd-4dee-8bd7-84715c2fd477 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:01.483877 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-6e78acaa-44b3-4311-bf8b-4fe050c9ba0f admin admin] The token cache is being invalidate because project 6da1fab8502c40b4b5edccb5e97e77c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:56:01.484169 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-6e78acaa-44b3-4311-bf8b-4fe050c9ba0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:01.484924 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-6e78acaa-44b3-4311-bf8b-4fe050c9ba0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:01.487416 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1003/2018] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:56:01 2022] DELETE /identity/v3/projects/6da1fab8502c40b4b5edccb5e97e77c2 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:56:01.492506 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88acf3ee-baca-4eba-9ec4-ca5ddcf9abc5 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:01.492860 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88acf3ee-baca-4eba-9ec4-ca5ddcf9abc5 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:01.493143 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88acf3ee-baca-4eba-9ec4-ca5ddcf9abc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:01.516056 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-88acf3ee-baca-4eba-9ec4-ca5ddcf9abc5 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:01.518226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-88acf3ee-baca-4eba-9ec4-ca5ddcf9abc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:01.525279 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1004/2019] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:01 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:01.533356 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11a8e3-d32d-4f39-b84d-9e5db769b9c2 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:01.533644 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11a8e3-d32d-4f39-b84d-9e5db769b9c2 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:01.533919 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d11a8e3-d32d-4f39-b84d-9e5db769b9c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:01.554908 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-9d11a8e3-d32d-4f39-b84d-9e5db769b9c2 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:01.556341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-9d11a8e3-d32d-4f39-b84d-9e5db769b9c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:01.565586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1005/2020] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:01 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:56:01.569847 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11a8e3-d32d-4f39-b84d-9e5db769b9c2 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:01.580573 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d11a8e3-d32d-4f39-b84d-9e5db769b9c2 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:01.604042 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e755801c-36bd-4dee-8bd7-84715c2fd477 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb1ca0c45c604b5ea6c6e621d332534f', 'request_id': 'req-e755801c-36bd-4dee-8bd7-84715c2fd477'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:01.615606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e755801c-36bd-4dee-8bd7-84715c2fd477 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb1ca0c45c604b5ea6c6e621d332534f', 'request_id': 'req-e755801c-36bd-4dee-8bd7-84715c2fd477'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:01.650400 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0df336-f782-4178-b32d-84cbe62cbd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:01.651033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0df336-f782-4178-b32d-84cbe62cbd0e admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:01.651345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0df336-f782-4178-b32d-84cbe62cbd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:01.651624 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0df336-f782-4178-b32d-84cbe62cbd0e admin admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:01.652387 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0df336-f782-4178-b32d-84cbe62cbd0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:56:01.652689 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0df336-f782-4178-b32d-84cbe62cbd0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:01.654449 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0df336-f782-4178-b32d-84cbe62cbd0e admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:01.671108 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1006/2021] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:56:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:56:01.672951 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0df336-f782-4178-b32d-84cbe62cbd0e admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:01.689293 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0df336-f782-4178-b32d-84cbe62cbd0e admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:01.694532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74e275-ff18-4f9d-9919-145105692843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:01.695249 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74e275-ff18-4f9d-9919-145105692843 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:01.695532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74e275-ff18-4f9d-9919-145105692843 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:01.695810 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec74e275-ff18-4f9d-9919-145105692843 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:01.696480 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74e275-ff18-4f9d-9919-145105692843 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:01.698459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec74e275-ff18-4f9d-9919-145105692843 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:01.754820 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e755801c-36bd-4dee-8bd7-84715c2fd477 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb1ca0c45c604b5ea6c6e621d332534f', 'request_id': 'req-e755801c-36bd-4dee-8bd7-84715c2fd477'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:01.767586 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e755801c-36bd-4dee-8bd7-84715c2fd477 admin admin] The token cache is being invalidate because project bb1ca0c45c604b5ea6c6e621d332534f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:56:01.767878 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e755801c-36bd-4dee-8bd7-84715c2fd477 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:01.768532 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e755801c-36bd-4dee-8bd7-84715c2fd477 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:01.770111 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1016/2022] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:56:01 2022] DELETE /identity/v3/projects/bb1ca0c45c604b5ea6c6e621d332534f => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:01.776030 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93106514-157c-49f3-9f87-2d26c601f973 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:01.776294 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93106514-157c-49f3-9f87-2d26c601f973 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:01.776569 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93106514-157c-49f3-9f87-2d26c601f973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:01.784626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:01.801290 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-93106514-157c-49f3-9f87-2d26c601f973 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:01.802506 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-93106514-157c-49f3-9f87-2d26c601f973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:01.812038 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1017/2023] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:01 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:56:01.820404 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8081c59-8ba6-40d2-be9c-b3ae322d1a13 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:01.820827 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8081c59-8ba6-40d2-be9c-b3ae322d1a13 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:01.821108 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8081c59-8ba6-40d2-be9c-b3ae322d1a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:01.828555 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1007/2024] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:56:01 2022] POST /identity/v3/projects => generated 374 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:56:01.832490 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74e275-ff18-4f9d-9919-145105692843 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:01.835553 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-ec74e275-ff18-4f9d-9919-145105692843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:01.852398 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-e8081c59-8ba6-40d2-be9c-b3ae322d1a13 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:01.853671 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-e8081c59-8ba6-40d2-be9c-b3ae322d1a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:01.939305 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec74e275-ff18-4f9d-9919-145105692843 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:01.947377 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca458341-332f-48d8-a527-f84bf297b9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:01.948502 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca458341-332f-48d8-a527-f84bf297b9ed admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:01.948502 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca458341-332f-48d8-a527-f84bf297b9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:01.949706 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca458341-332f-48d8-a527-f84bf297b9ed admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:01.950605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca458341-332f-48d8-a527-f84bf297b9ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:01.952932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca458341-332f-48d8-a527-f84bf297b9ed admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:01.967356 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1018/2025] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:01 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:56:01.969345 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8081c59-8ba6-40d2-be9c-b3ae322d1a13 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:01.983650 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8081c59-8ba6-40d2-be9c-b3ae322d1a13 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:01.991718 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7bdc2-ef4a-40dd-98b7-097b559f4ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:01.992571 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7bdc2-ef4a-40dd-98b7-097b559f4ade admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:01.993041 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7bdc2-ef4a-40dd-98b7-097b559f4ade admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:01.993461 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c7bdc2-ef4a-40dd-98b7-097b559f4ade admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:01.994305 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c7bdc2-ef4a-40dd-98b7-097b559f4ade admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:56:01.994780 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c7bdc2-ef4a-40dd-98b7-097b559f4ade admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:01.997615 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c7bdc2-ef4a-40dd-98b7-097b559f4ade admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:02.015222 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1019/2026] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:56:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:56:02.019119 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7bdc2-ef4a-40dd-98b7-097b559f4ade admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:02.032157 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c7bdc2-ef4a-40dd-98b7-097b559f4ade admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:02.038442 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35348b7f-7d36-4f10-a5c7-de9c66820bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:02.039792 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35348b7f-7d36-4f10-a5c7-de9c66820bbd admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:02.039792 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35348b7f-7d36-4f10-a5c7-de9c66820bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:02.040848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35348b7f-7d36-4f10-a5c7-de9c66820bbd admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:02.041854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35348b7f-7d36-4f10-a5c7-de9c66820bbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:02.043971 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35348b7f-7d36-4f10-a5c7-de9c66820bbd admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:02.109874 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1008/2027] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:56:01 2022] POST /identity/v3/users => generated 288 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:56:02.111922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51541ef7-11ca-48a0-a818-9544227128c5 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:02.112422 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51541ef7-11ca-48a0-a818-9544227128c5 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:02.112864 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51541ef7-11ca-48a0-a818-9544227128c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:02.160826 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:02.167771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1020/2028] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:56:02 2022] POST /identity/v3/projects => generated 362 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:56:02.168770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35348b7f-7d36-4f10-a5c7-de9c66820bbd admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:02.174026 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-51541ef7-11ca-48a0-a818-9544227128c5 None None] MFA Rules not processed for user `bec9ea3f61b7483da3a629b290000fec`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:02.176015 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-51541ef7-11ca-48a0-a818-9544227128c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:02.253732 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35348b7f-7d36-4f10-a5c7-de9c66820bbd admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:02.259785 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb92bed-27be-4fd5-acfb-93457f1a63ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:02.260778 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb92bed-27be-4fd5-acfb-93457f1a63ca admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:02.260898 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb92bed-27be-4fd5-acfb-93457f1a63ca admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:02.261319 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb92bed-27be-4fd5-acfb-93457f1a63ca admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:02.262844 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb92bed-27be-4fd5-acfb-93457f1a63ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:02.265016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb92bed-27be-4fd5-acfb-93457f1a63ca admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:02.286205 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1021/2029] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:02 2022] GET /identity/v3/roles => generated 1427 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:56:02.286205 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb92bed-27be-4fd5-acfb-93457f1a63ca admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:02.296566 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb92bed-27be-4fd5-acfb-93457f1a63ca admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:02.302198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096a29f-1fd8-4402-8ecb-90aa217a3c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:02.302881 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096a29f-1fd8-4402-8ecb-90aa217a3c96 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:02.303242 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096a29f-1fd8-4402-8ecb-90aa217a3c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:02.303523 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0096a29f-1fd8-4402-8ecb-90aa217a3c96 admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:02.304198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096a29f-1fd8-4402-8ecb-90aa217a3c96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:02.306210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0096a29f-1fd8-4402-8ecb-90aa217a3c96 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:02.307463 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1009/2030] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:02.309215 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51541ef7-11ca-48a0-a818-9544227128c5 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:02.321260 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51541ef7-11ca-48a0-a818-9544227128c5 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:02.328043 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80012c21-d99b-41f8-80dd-a03e1ed10a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:02.328970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80012c21-d99b-41f8-80dd-a03e1ed10a9e admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:02.329287 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80012c21-d99b-41f8-80dd-a03e1ed10a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:02.335175 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80012c21-d99b-41f8-80dd-a03e1ed10a9e admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:02.335175 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80012c21-d99b-41f8-80dd-a03e1ed10a9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:02.335175 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80012c21-d99b-41f8-80dd-a03e1ed10a9e admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:02.347157 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1010/2031] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:02 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:56:02.347157 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80012c21-d99b-41f8-80dd-a03e1ed10a9e admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:02.347658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-80012c21-d99b-41f8-80dd-a03e1ed10a9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:02.361144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80012c21-d99b-41f8-80dd-a03e1ed10a9e admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:02.366056 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191af8b1-eb5b-4a1c-98f1-56eb39e226e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:02.366660 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191af8b1-eb5b-4a1c-98f1-56eb39e226e9 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:02.368158 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191af8b1-eb5b-4a1c-98f1-56eb39e226e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:02.368158 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191af8b1-eb5b-4a1c-98f1-56eb39e226e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:02.368158 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191af8b1-eb5b-4a1c-98f1-56eb39e226e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:02.370226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191af8b1-eb5b-4a1c-98f1-56eb39e226e9 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:02.408051 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1022/2032] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:56:02 2022] POST /identity/v3/users => generated 282 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:56:02.408270 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096a29f-1fd8-4402-8ecb-90aa217a3c96 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:02.418821 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0096a29f-1fd8-4402-8ecb-90aa217a3c96 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:02.423789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae765d0-ba20-4aa1-9015-bb74f1717780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:02.424516 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae765d0-ba20-4aa1-9015-bb74f1717780 admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:02.424807 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae765d0-ba20-4aa1-9015-bb74f1717780 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:02.425662 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae765d0-ba20-4aa1-9015-bb74f1717780 admin admin] PATH_INFO: `/v3/projects/6e8a725726a549cfa6873fa0571062c2/users/5e0379bb8134440aa57ff1e89f068a60/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:02.442010 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae765d0-ba20-4aa1-9015-bb74f1717780 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e8a725726a549cfa6873fa0571062c2, user_id=5e0379bb8134440aa57ff1e89f068a60, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:02.444796 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae765d0-ba20-4aa1-9015-bb74f1717780 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:02.475450 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1011/2033] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:56:02 2022] POST /identity/v3/projects => generated 372 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:56:02.477116 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191af8b1-eb5b-4a1c-98f1-56eb39e226e9 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:02.479073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-191af8b1-eb5b-4a1c-98f1-56eb39e226e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:02.538509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1023/2034] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:56:02 2022] PUT /identity/v3/projects/6e8a725726a549cfa6873fa0571062c2/users/5e0379bb8134440aa57ff1e89f068a60/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:02.539546 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae765d0-ba20-4aa1-9015-bb74f1717780 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:02.555507 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191af8b1-eb5b-4a1c-98f1-56eb39e226e9 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:02.621224 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae765d0-ba20-4aa1-9015-bb74f1717780 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:02.626884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4098d1-2950-4917-890e-a5f04f5d9439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:02.627650 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4098d1-2950-4917-890e-a5f04f5d9439 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:02.627788 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc331f-5eaf-40f8-94f7-2a50db192c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:02.627898 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4098d1-2950-4917-890e-a5f04f5d9439 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:02.628102 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4098d1-2950-4917-890e-a5f04f5d9439 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:02.628231 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc331f-5eaf-40f8-94f7-2a50db192c53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:02.628738 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc331f-5eaf-40f8-94f7-2a50db192c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:02.628862 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fc331f-5eaf-40f8-94f7-2a50db192c53 admin admin] PATH_INFO: `/v3/users/41b74e1893f84606966996e2c0ef31e7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:02.628952 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4098d1-2950-4917-890e-a5f04f5d9439 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:02.630113 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc331f-5eaf-40f8-94f7-2a50db192c53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41b74e1893f84606966996e2c0ef31e7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:02.630441 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4098d1-2950-4917-890e-a5f04f5d9439 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:02.632278 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fc331f-5eaf-40f8-94f7-2a50db192c53 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:02.642414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1024/2035] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:02 2022] GET /identity/v3/roles => generated 1427 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:56:02.646194 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4098d1-2950-4917-890e-a5f04f5d9439 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:02.657757 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4098d1-2950-4917-890e-a5f04f5d9439 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:02.666606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ff4b7c-2209-4359-92d5-0575a4628b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:02.666606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff4b7c-2209-4359-92d5-0575a4628b14 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:02.666606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff4b7c-2209-4359-92d5-0575a4628b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:02.666606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ff4b7c-2209-4359-92d5-0575a4628b14 admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:02.667166 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff4b7c-2209-4359-92d5-0575a4628b14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:02.669502 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ff4b7c-2209-4359-92d5-0575a4628b14 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:02.831330 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:02.838748 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1025/2036] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:56:02 2022] POST /identity/v3/users => generated 293 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:56:02.840436 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eae88e-019e-4ec5-ad6e-4ad6c063e11c None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:02.841014 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eae88e-019e-4ec5-ad6e-4ad6c063e11c None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:02.841014 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eae88e-019e-4ec5-ad6e-4ad6c063e11c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:02.878406 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-d1eae88e-019e-4ec5-ad6e-4ad6c063e11c None None] MFA Rules not processed for user `5e0379bb8134440aa57ff1e89f068a60`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:02.880185 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-d1eae88e-019e-4ec5-ad6e-4ad6c063e11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:02.901215 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-f3fc331f-5eaf-40f8-94f7-2a50db192c53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41b74e1893f84606966996e2c0ef31e7'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:02.909493 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-f3fc331f-5eaf-40f8-94f7-2a50db192c53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41b74e1893f84606966996e2c0ef31e7'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:02.910695 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-f3fc331f-5eaf-40f8-94f7-2a50db192c53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41b74e1893f84606966996e2c0ef31e7'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:02.960732 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-f3fc331f-5eaf-40f8-94f7-2a50db192c53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41b74e1893f84606966996e2c0ef31e7'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:02.966002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1026/2037] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:02.970394 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eae88e-019e-4ec5-ad6e-4ad6c063e11c None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:02.975179 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-f3fc331f-5eaf-40f8-94f7-2a50db192c53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41b74e1893f84606966996e2c0ef31e7'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:02.980559 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eae88e-019e-4ec5-ad6e-4ad6c063e11c None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:02.983362 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1012/2038] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:56:02 2022] DELETE /identity/v3/users/41b74e1893f84606966996e2c0ef31e7 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:02.984194 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc331f-5eaf-40f8-94f7-2a50db192c53 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:02.985734 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f3fc331f-5eaf-40f8-94f7-2a50db192c53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:03.046154 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b9bd9a-1ee3-478a-973b-e66a17e139cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:03.046785 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9bd9a-1ee3-478a-973b-e66a17e139cd admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:03.047096 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9bd9a-1ee3-478a-973b-e66a17e139cd admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:03.047372 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9bd9a-1ee3-478a-973b-e66a17e139cd admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:03.048064 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b9bd9a-1ee3-478a-973b-e66a17e139cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:03.049657 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b9bd9a-1ee3-478a-973b-e66a17e139cd admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:03.052006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fc331f-5eaf-40f8-94f7-2a50db192c53 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:03.056682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9340f6a-b0d1-4408-94db-d42b8cc29fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:03.057304 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9340f6a-b0d1-4408-94db-d42b8cc29fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:03.057588 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9340f6a-b0d1-4408-94db-d42b8cc29fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:03.057861 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9340f6a-b0d1-4408-94db-d42b8cc29fb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:03.058557 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9340f6a-b0d1-4408-94db-d42b8cc29fb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:03.061075 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9340f6a-b0d1-4408-94db-d42b8cc29fb1 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:03.061223 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1027/2039] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:02 2022] GET /identity/v3/roles => generated 1427 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:56:03.062227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b9bd9a-1ee3-478a-973b-e66a17e139cd admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:03.064786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:03.066183 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-97b9bd9a-1ee3-478a-973b-e66a17e139cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:03.074657 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1013/2040] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:02 2022] GET /identity/v3/roles => generated 1427 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:56:03.076846 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53565f-086a-47d3-a0e2-7734521b6cbb None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:03.077132 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53565f-086a-47d3-a0e2-7734521b6cbb None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:03.077408 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc53565f-086a-47d3-a0e2-7734521b6cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:03.095181 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:03.097922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-bc53565f-086a-47d3-a0e2-7734521b6cbb None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:03.099172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-bc53565f-086a-47d3-a0e2-7734521b6cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:03.107526 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1014/2041] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:03 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:03.108095 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53565f-086a-47d3-a0e2-7734521b6cbb None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:03.112075 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-bc53565f-086a-47d3-a0e2-7734521b6cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:03.123888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc53565f-086a-47d3-a0e2-7734521b6cbb None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:03.129953 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa0e89-f767-48d1-a263-0bf08ff52ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:03.130754 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa0e89-f767-48d1-a263-0bf08ff52ec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:03.131071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa0e89-f767-48d1-a263-0bf08ff52ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:03.131347 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa0e89-f767-48d1-a263-0bf08ff52ec2 admin admin] PATH_INFO: `/v3/projects/704489e9274f49a38932013453bbd41d/users/3ddb02b56590442488b97f7defe51faf/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:03.134842 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b9bd9a-1ee3-478a-973b-e66a17e139cd admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:03.139170 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e89da1d-bcc1-4fc6-a201-8f8b27a40d2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:03.139835 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89da1d-bcc1-4fc6-a201-8f8b27a40d2b service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:03.140111 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89da1d-bcc1-4fc6-a201-8f8b27a40d2b service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:03.140381 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89da1d-bcc1-4fc6-a201-8f8b27a40d2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:03.142160 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-3e89da1d-bcc1-4fc6-a201-8f8b27a40d2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:03.149262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:03.152469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fa0e89-f767-48d1-a263-0bf08ff52ec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=704489e9274f49a38932013453bbd41d, user_id=3ddb02b56590442488b97f7defe51faf, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:03.156707 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fa0e89-f767-48d1-a263-0bf08ff52ec2 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:03.220170 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e89da1d-bcc1-4fc6-a201-8f8b27a40d2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:03.222258 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e89da1d-bcc1-4fc6-a201-8f8b27a40d2b service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:03.233162 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1028/2042] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:56:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:03.233990 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e89da1d-bcc1-4fc6-a201-8f8b27a40d2b service cinder] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:03.244996 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e89da1d-bcc1-4fc6-a201-8f8b27a40d2b service cinder] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:03.247917 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1015/2043] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:56:03 2022] PUT /identity/v3/projects/704489e9274f49a38932013453bbd41d/users/3ddb02b56590442488b97f7defe51faf/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:03.248527 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa0e89-f767-48d1-a263-0bf08ff52ec2 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:03.250859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-88fa0e89-f767-48d1-a263-0bf08ff52ec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:03.329485 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea3baa1-6541-4684-ad1a-c9ab427d5834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:03.330616 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3baa1-6541-4684-ad1a-c9ab427d5834 admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:03.331204 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3baa1-6541-4684-ad1a-c9ab427d5834 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:03.331315 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea3baa1-6541-4684-ad1a-c9ab427d5834 admin admin] PATH_INFO: `/v3/projects/1c276ee2728944f3987c76ccbe02a0d0/users/7be4e9c4be7b49d08923f0f9c4b7dca0/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:03.340238 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa0e89-f767-48d1-a263-0bf08ff52ec2 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:03.352688 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea3baa1-6541-4684-ad1a-c9ab427d5834 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c276ee2728944f3987c76ccbe02a0d0, user_id=7be4e9c4be7b49d08923f0f9c4b7dca0, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:03.353669 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fe8141-5a31-40d7-902f-f34db72dd9a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:03.354558 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe8141-5a31-40d7-902f-f34db72dd9a2 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:03.354704 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe8141-5a31-40d7-902f-f34db72dd9a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:03.354928 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fe8141-5a31-40d7-902f-f34db72dd9a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:03.356141 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea3baa1-6541-4684-ad1a-c9ab427d5834 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:03.369726 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fe8141-5a31-40d7-902f-f34db72dd9a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:03.371301 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fe8141-5a31-40d7-902f-f34db72dd9a2 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:03.389454 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1016/2044] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:56:03 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:03.391859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c4fc2-c939-4c32-8ab6-a40291295f31 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:03.392308 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c4fc2-c939-4c32-8ab6-a40291295f31 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:03.392786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c4fc2-c939-4c32-8ab6-a40291295f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:03.444316 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1029/2045] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:56:03 2022] PUT /identity/v3/projects/1c276ee2728944f3987c76ccbe02a0d0/users/7be4e9c4be7b49d08923f0f9c4b7dca0/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:03.448345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-249c4fc2-c939-4c32-8ab6-a40291295f31 None None] MFA Rules not processed for user `3ddb02b56590442488b97f7defe51faf`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:03.448779 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea3baa1-6541-4684-ad1a-c9ab427d5834 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:03.451783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-249c4fc2-c939-4c32-8ab6-a40291295f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:03.540588 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:03.569328 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea3baa1-6541-4684-ad1a-c9ab427d5834 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:03.578510 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68d5bb-b061-4776-8194-03508cec9647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:03.579307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68d5bb-b061-4776-8194-03508cec9647 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:03.579731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68d5bb-b061-4776-8194-03508cec9647 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:03.580804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd68d5bb-b061-4776-8194-03508cec9647 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:03.581180 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd68d5bb-b061-4776-8194-03508cec9647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:03.583634 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd68d5bb-b061-4776-8194-03508cec9647 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:03.590481 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1017/2046] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:03.599083 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1030/2047] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:03 2022] GET /identity/v3/roles => generated 1427 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:56:03.604194 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c4fc2-c939-4c32-8ab6-a40291295f31 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:03.627484 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c4fc2-c939-4c32-8ab6-a40291295f31 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:03.629492 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35273e-9bf8-4ce7-a87f-e00b621dcea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:03.630367 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35273e-9bf8-4ce7-a87f-e00b621dcea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:03.630720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35273e-9bf8-4ce7-a87f-e00b621dcea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:03.631091 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35273e-9bf8-4ce7-a87f-e00b621dcea7 admin admin] PATH_INFO: `/v3/projects/1c276ee2728944f3987c76ccbe02a0d0/users/7be4e9c4be7b49d08923f0f9c4b7dca0/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:03.633161 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd35273e-9bf8-4ce7-a87f-e00b621dcea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c276ee2728944f3987c76ccbe02a0d0, user_id=7be4e9c4be7b49d08923f0f9c4b7dca0, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:03.635960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd35273e-9bf8-4ce7-a87f-e00b621dcea7 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:03.637439 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68d5bb-b061-4776-8194-03508cec9647 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:03.640750 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-cd68d5bb-b061-4776-8194-03508cec9647 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:03.648059 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1018/2048] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:56:03 2022] PUT /identity/v3/projects/1c276ee2728944f3987c76ccbe02a0d0/users/7be4e9c4be7b49d08923f0f9c4b7dca0/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:03.714977 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd68d5bb-b061-4776-8194-03508cec9647 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:03.771925 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ced8c5-5227-42c0-9232-885978641ec1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:03.772564 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ced8c5-5227-42c0-9232-885978641ec1 service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:03.772839 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ced8c5-5227-42c0-9232-885978641ec1 service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:03.773114 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ced8c5-5227-42c0-9232-885978641ec1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:03.877259 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ced8c5-5227-42c0-9232-885978641ec1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:03.879397 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ced8c5-5227-42c0-9232-885978641ec1 service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:03.891828 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1031/2049] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:56:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:56:03.908896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35273e-9bf8-4ce7-a87f-e00b621dcea7 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:03.939823 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:03.954100 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ced8c5-5227-42c0-9232-885978641ec1 service cinder] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:03.954100 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-69ced8c5-5227-42c0-9232-885978641ec1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:04.009644 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35273e-9bf8-4ce7-a87f-e00b621dcea7 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:04.017461 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e87f1b9-44dd-42bc-8296-d14643877c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:04.018458 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87f1b9-44dd-42bc-8296-d14643877c47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:04.018927 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87f1b9-44dd-42bc-8296-d14643877c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:04.019433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e87f1b9-44dd-42bc-8296-d14643877c47 admin admin] PATH_INFO: `/v3/users/3ddb02b56590442488b97f7defe51faf` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:04.021813 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e87f1b9-44dd-42bc-8296-d14643877c47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ddb02b56590442488b97f7defe51faf)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:04.025607 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e87f1b9-44dd-42bc-8296-d14643877c47 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:04.082465 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ced8c5-5227-42c0-9232-885978641ec1 service cinder] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:04.087338 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962c257-149c-4382-a0b7-9be0294a0401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:04.087994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962c257-149c-4382-a0b7-9be0294a0401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:04.088263 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962c257-149c-4382-a0b7-9be0294a0401 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:04.088534 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b962c257-149c-4382-a0b7-9be0294a0401 admin admin] PATH_INFO: `/v3/projects/f7e87b8d85ce4072ba8fc7caa8144af6` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:04.089763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b962c257-149c-4382-a0b7-9be0294a0401 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7e87b8d85ce4072ba8fc7caa8144af6)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:04.092558 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b962c257-149c-4382-a0b7-9be0294a0401 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:04.099710 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:04.261651 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-b962c257-149c-4382-a0b7-9be0294a0401 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7e87b8d85ce4072ba8fc7caa8144af6', 'request_id': 'req-b962c257-149c-4382-a0b7-9be0294a0401'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:04.263881 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2e87f1b9-44dd-42bc-8296-d14643877c47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ddb02b56590442488b97f7defe51faf'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:04.268797 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-b962c257-149c-4382-a0b7-9be0294a0401 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7e87b8d85ce4072ba8fc7caa8144af6', 'request_id': 'req-b962c257-149c-4382-a0b7-9be0294a0401'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:04.272979 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2e87f1b9-44dd-42bc-8296-d14643877c47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ddb02b56590442488b97f7defe51faf'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:04.273790 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2e87f1b9-44dd-42bc-8296-d14643877c47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ddb02b56590442488b97f7defe51faf'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:04.372959 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-b962c257-149c-4382-a0b7-9be0294a0401 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7e87b8d85ce4072ba8fc7caa8144af6', 'request_id': 'req-b962c257-149c-4382-a0b7-9be0294a0401'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:04.385792 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-b962c257-149c-4382-a0b7-9be0294a0401 admin admin] The token cache is being invalidate because project f7e87b8d85ce4072ba8fc7caa8144af6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:56:04.386085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-b962c257-149c-4382-a0b7-9be0294a0401 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:04.386728 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-b962c257-149c-4382-a0b7-9be0294a0401 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:04.388504 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1032/2050] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:56:03 2022] DELETE /identity/v3/projects/f7e87b8d85ce4072ba8fc7caa8144af6 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:56:04.392452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962c257-149c-4382-a0b7-9be0294a0401 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:04.394061 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-b962c257-149c-4382-a0b7-9be0294a0401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:04.435019 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2e87f1b9-44dd-42bc-8296-d14643877c47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ddb02b56590442488b97f7defe51faf'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:04.458683 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2e87f1b9-44dd-42bc-8296-d14643877c47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ddb02b56590442488b97f7defe51faf'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:04.469734 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1019/2051] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:56:03 2022] DELETE /identity/v3/users/3ddb02b56590442488b97f7defe51faf => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:04.476343 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cab02-978c-4fee-bc88-bf6c3b301c5f None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:04.476642 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cab02-978c-4fee-bc88-bf6c3b301c5f None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:04.476973 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97cab02-978c-4fee-bc88-bf6c3b301c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:04.483427 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b962c257-149c-4382-a0b7-9be0294a0401 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:04.501044 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-a97cab02-978c-4fee-bc88-bf6c3b301c5f None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:04.502474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-a97cab02-978c-4fee-bc88-bf6c3b301c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:04.554804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf957b71-b19c-4bcc-b9c9-2a26d436408f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:04.555839 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf957b71-b19c-4bcc-b9c9-2a26d436408f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:04.556338 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf957b71-b19c-4bcc-b9c9-2a26d436408f admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:04.556765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf957b71-b19c-4bcc-b9c9-2a26d436408f admin admin] PATH_INFO: `/v3/users/13efd1206d3c42e4ae773b6ae8a306db` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:04.558417 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf957b71-b19c-4bcc-b9c9-2a26d436408f admin admin] RBAC: Authorizing `identity:delete_user(user_id=13efd1206d3c42e4ae773b6ae8a306db)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:04.561405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf957b71-b19c-4bcc-b9c9-2a26d436408f admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:04.572914 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1020/2052] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:04 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:56:04.579593 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97cab02-978c-4fee-bc88-bf6c3b301c5f None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:04.581594 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-a97cab02-978c-4fee-bc88-bf6c3b301c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:04.664106 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:04.684635 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97cab02-978c-4fee-bc88-bf6c3b301c5f None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:04.693340 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbad4db-85b1-4a83-abd1-b6908ab952e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:04.694520 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbad4db-85b1-4a83-abd1-b6908ab952e5 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:04.697092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbad4db-85b1-4a83-abd1-b6908ab952e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:04.697092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbad4db-85b1-4a83-abd1-b6908ab952e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:04.710193 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbad4db-85b1-4a83-abd1-b6908ab952e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:04.712149 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbad4db-85b1-4a83-abd1-b6908ab952e5 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:04.723487 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1021/2053] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:56:04 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:04.909391 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:04.911879 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-bf957b71-b19c-4bcc-b9c9-2a26d436408f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13efd1206d3c42e4ae773b6ae8a306db'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:04.912836 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-bf957b71-b19c-4bcc-b9c9-2a26d436408f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13efd1206d3c42e4ae773b6ae8a306db'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:05.257921 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-bf957b71-b19c-4bcc-b9c9-2a26d436408f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13efd1206d3c42e4ae773b6ae8a306db'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:05.266788 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-bf957b71-b19c-4bcc-b9c9-2a26d436408f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13efd1206d3c42e4ae773b6ae8a306db'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:05.279029 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-bf957b71-b19c-4bcc-b9c9-2a26d436408f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13efd1206d3c42e4ae773b6ae8a306db'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:05.285662 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1033/2054] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:56:04 2022] DELETE /identity/v3/users/13efd1206d3c42e4ae773b6ae8a306db => generated 0 bytes in 894 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:56:05.364999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbad4db-85b1-4a83-abd1-b6908ab952e5 service neutron] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:05.366896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-7fbad4db-85b1-4a83-abd1-b6908ab952e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:05.405594 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbad4db-85b1-4a83-abd1-b6908ab952e5 service neutron] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:05.525699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320ba145-2432-4465-bd1b-29a1d4c9ed65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:05.530970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320ba145-2432-4465-bd1b-29a1d4c9ed65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:05.530970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320ba145-2432-4465-bd1b-29a1d4c9ed65 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:05.530970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320ba145-2432-4465-bd1b-29a1d4c9ed65 admin admin] PATH_INFO: `/v3/projects/704489e9274f49a38932013453bbd41d` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:05.530970 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320ba145-2432-4465-bd1b-29a1d4c9ed65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=704489e9274f49a38932013453bbd41d)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:05.532603 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320ba145-2432-4465-bd1b-29a1d4c9ed65 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:05.603620 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90dbbe-eeed-4bf9-9076-a5441962b26a None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:05.604152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90dbbe-eeed-4bf9-9076-a5441962b26a None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:05.604492 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe90dbbe-eeed-4bf9-9076-a5441962b26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:05.688682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-320ba145-2432-4465-bd1b-29a1d4c9ed65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '704489e9274f49a38932013453bbd41d', 'request_id': 'req-320ba145-2432-4465-bd1b-29a1d4c9ed65'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:05.697594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-fe90dbbe-eeed-4bf9-9076-a5441962b26a None None] MFA Rules not processed for user `7be4e9c4be7b49d08923f0f9c4b7dca0`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:05.699409 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-fe90dbbe-eeed-4bf9-9076-a5441962b26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:05.765031 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-320ba145-2432-4465-bd1b-29a1d4c9ed65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '704489e9274f49a38932013453bbd41d', 'request_id': 'req-320ba145-2432-4465-bd1b-29a1d4c9ed65'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:05.772649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-320ba145-2432-4465-bd1b-29a1d4c9ed65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '704489e9274f49a38932013453bbd41d', 'request_id': 'req-320ba145-2432-4465-bd1b-29a1d4c9ed65'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:05.788403 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-320ba145-2432-4465-bd1b-29a1d4c9ed65 admin admin] The token cache is being invalidate because project 704489e9274f49a38932013453bbd41d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:56:05.789100 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-320ba145-2432-4465-bd1b-29a1d4c9ed65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:05.789993 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-320ba145-2432-4465-bd1b-29a1d4c9ed65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:05.794167 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1022/2055] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:56:05 2022] DELETE /identity/v3/projects/704489e9274f49a38932013453bbd41d => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:05.801672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e4247-41f4-4704-9a34-fc2c0df3d6ac None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:05.802195 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e4247-41f4-4704-9a34-fc2c0df3d6ac None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:05.802695 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670e4247-41f4-4704-9a34-fc2c0df3d6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:05.811385 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1034/2056] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:05 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:56:05.823176 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90dbbe-eeed-4bf9-9076-a5441962b26a None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:05.827328 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-fe90dbbe-eeed-4bf9-9076-a5441962b26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:05.882205 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-670e4247-41f4-4704-9a34-fc2c0df3d6ac None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:05.883550 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-670e4247-41f4-4704-9a34-fc2c0df3d6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:05.905822 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe90dbbe-eeed-4bf9-9076-a5441962b26a None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:05.908008 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a21e4-c2cc-46f9-86f3-9591c4a86485 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:05.908821 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a21e4-c2cc-46f9-86f3-9591c4a86485 service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:05.909196 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a21e4-c2cc-46f9-86f3-9591c4a86485 service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:05.909546 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a21e4-c2cc-46f9-86f3-9591c4a86485 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:05.913905 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-841a21e4-c2cc-46f9-86f3-9591c4a86485 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:05.947786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1023/2057] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:56:05.956455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e4247-41f4-4704-9a34-fc2c0df3d6ac None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:05.956455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-670e4247-41f4-4704-9a34-fc2c0df3d6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:05.963722 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670e4247-41f4-4704-9a34-fc2c0df3d6ac None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:05.968857 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748036eb-daee-489d-bcb5-4c67188b2c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:05.969518 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748036eb-daee-489d-bcb5-4c67188b2c58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:05.969802 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748036eb-daee-489d-bcb5-4c67188b2c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:05.970083 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748036eb-daee-489d-bcb5-4c67188b2c58 admin admin] PATH_INFO: `/v3/projects/525584e541654e8788d53de31d384d34` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:05.971347 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748036eb-daee-489d-bcb5-4c67188b2c58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=525584e541654e8788d53de31d384d34)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:05.973393 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748036eb-daee-489d-bcb5-4c67188b2c58 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:05.982820 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a21e4-c2cc-46f9-86f3-9591c4a86485 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:05.984324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841a21e4-c2cc-46f9-86f3-9591c4a86485 service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:05.994820 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1035/2058] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:56:05 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:56:05.997632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa0573-8542-4a59-894d-400136b4fb9f None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:05.997632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa0573-8542-4a59-894d-400136b4fb9f None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:05.997632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49aa0573-8542-4a59-894d-400136b4fb9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:06.020844 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-49aa0573-8542-4a59-894d-400136b4fb9f None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:06.020844 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-49aa0573-8542-4a59-894d-400136b4fb9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:06.030374 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1036/2059] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:06.034869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa0573-8542-4a59-894d-400136b4fb9f None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:06.047716 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49aa0573-8542-4a59-894d-400136b4fb9f None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:06.054154 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460ce5c-4eec-4ce3-b4d8-f33eb5679e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:06.054966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460ce5c-4eec-4ce3-b4d8-f33eb5679e03 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:06.055911 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460ce5c-4eec-4ce3-b4d8-f33eb5679e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:06.056193 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460ce5c-4eec-4ce3-b4d8-f33eb5679e03 admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:06.056907 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2460ce5c-4eec-4ce3-b4d8-f33eb5679e03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:56:06.057216 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2460ce5c-4eec-4ce3-b4d8-f33eb5679e03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:06.059000 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2460ce5c-4eec-4ce3-b4d8-f33eb5679e03 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:06.075094 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1037/2060] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:56:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:56:06.078919 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460ce5c-4eec-4ce3-b4d8-f33eb5679e03 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:06.089974 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460ce5c-4eec-4ce3-b4d8-f33eb5679e03 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:06.103162 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-748036eb-daee-489d-bcb5-4c67188b2c58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '525584e541654e8788d53de31d384d34', 'request_id': 'req-748036eb-daee-489d-bcb5-4c67188b2c58'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:06.106591 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:06.155594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c473075-0580-486f-971e-96b4313515b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:06.156366 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c473075-0580-486f-971e-96b4313515b0 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:06.156885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c473075-0580-486f-971e-96b4313515b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:06.157228 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c473075-0580-486f-971e-96b4313515b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:06.158277 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c473075-0580-486f-971e-96b4313515b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:06.160336 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c473075-0580-486f-971e-96b4313515b0 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:06.166482 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-748036eb-daee-489d-bcb5-4c67188b2c58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '525584e541654e8788d53de31d384d34', 'request_id': 'req-748036eb-daee-489d-bcb5-4c67188b2c58'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:06.176866 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-748036eb-daee-489d-bcb5-4c67188b2c58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '525584e541654e8788d53de31d384d34', 'request_id': 'req-748036eb-daee-489d-bcb5-4c67188b2c58'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:06.195027 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-748036eb-daee-489d-bcb5-4c67188b2c58 admin admin] The token cache is being invalidate because project 525584e541654e8788d53de31d384d34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:56:06.195411 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-748036eb-daee-489d-bcb5-4c67188b2c58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:06.196062 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-748036eb-daee-489d-bcb5-4c67188b2c58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:06.197665 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1024/2061] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:56:05 2022] DELETE /identity/v3/projects/525584e541654e8788d53de31d384d34 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:06.203307 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620eaae7-5543-45eb-9ca8-448ac4dc7789 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:06.203595 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620eaae7-5543-45eb-9ca8-448ac4dc7789 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:06.203874 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620eaae7-5543-45eb-9ca8-448ac4dc7789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:06.224502 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-620eaae7-5543-45eb-9ca8-448ac4dc7789 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:06.225690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-620eaae7-5543-45eb-9ca8-448ac4dc7789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:06.234594 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1025/2062] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:56:06.242356 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376427bb-89bb-40bb-8bed-c33564653cd5 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:06.242640 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376427bb-89bb-40bb-8bed-c33564653cd5 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:06.242915 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376427bb-89bb-40bb-8bed-c33564653cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:06.263927 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-376427bb-89bb-40bb-8bed-c33564653cd5 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:06.265152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-376427bb-89bb-40bb-8bed-c33564653cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:06.275675 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1026/2063] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:06 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:56:06.284690 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1038/2064] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:56:06 2022] POST /identity/v3/projects => generated 370 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:56:06.287492 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376427bb-89bb-40bb-8bed-c33564653cd5 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:06.289059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c473075-0580-486f-971e-96b4313515b0 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:06.291837 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-3c473075-0580-486f-971e-96b4313515b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:06.303989 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376427bb-89bb-40bb-8bed-c33564653cd5 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:06.380348 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb989c04-f0ec-465e-a7aa-57cde1ab944b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:06.381576 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb989c04-f0ec-465e-a7aa-57cde1ab944b admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:06.382301 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb989c04-f0ec-465e-a7aa-57cde1ab944b admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:06.382871 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb989c04-f0ec-465e-a7aa-57cde1ab944b admin admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:06.384353 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb989c04-f0ec-465e-a7aa-57cde1ab944b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:56:06.384957 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb989c04-f0ec-465e-a7aa-57cde1ab944b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:06.387657 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb989c04-f0ec-465e-a7aa-57cde1ab944b admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:06.391959 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c473075-0580-486f-971e-96b4313515b0 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:06.402024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b921cd-ef9b-4858-83de-b6fd0dadea51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:06.402024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b921cd-ef9b-4858-83de-b6fd0dadea51 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:06.402024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b921cd-ef9b-4858-83de-b6fd0dadea51 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:06.402024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b921cd-ef9b-4858-83de-b6fd0dadea51 admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:06.402396 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b921cd-ef9b-4858-83de-b6fd0dadea51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:06.402396 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b921cd-ef9b-4858-83de-b6fd0dadea51 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:06.408446 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1027/2065] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:56:06 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:56:06.409806 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb989c04-f0ec-465e-a7aa-57cde1ab944b admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:06.421732 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb989c04-f0ec-465e-a7aa-57cde1ab944b admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:06.431836 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45718f46-7962-4dfd-b72c-c0561f50f135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:06.438526 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45718f46-7962-4dfd-b72c-c0561f50f135 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:06.438526 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45718f46-7962-4dfd-b72c-c0561f50f135 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:06.438526 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45718f46-7962-4dfd-b72c-c0561f50f135 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:06.438526 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45718f46-7962-4dfd-b72c-c0561f50f135 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:06.438526 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45718f46-7962-4dfd-b72c-c0561f50f135 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:06.532132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:06.567841 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1039/2066] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:56:06 2022] POST /identity/v3/users => generated 286 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:56:06.569778 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b921cd-ef9b-4858-83de-b6fd0dadea51 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:06.574613 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1028/2067] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:56:06 2022] POST /identity/v3/projects => generated 366 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:56:06.577861 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45718f46-7962-4dfd-b72c-c0561f50f135 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:06.613350 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b921cd-ef9b-4858-83de-b6fd0dadea51 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:06.695783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45718f46-7962-4dfd-b72c-c0561f50f135 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:06.700632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ab60d-b303-4ec1-9207-ee565b2165e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:06.700632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ab60d-b303-4ec1-9207-ee565b2165e6 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:06.700632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ab60d-b303-4ec1-9207-ee565b2165e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:06.700632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0ab60d-b303-4ec1-9207-ee565b2165e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:06.701189 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ab60d-b303-4ec1-9207-ee565b2165e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:06.702841 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0ab60d-b303-4ec1-9207-ee565b2165e6 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:06.704542 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e186e0-153d-4030-9024-c91301762a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:06.705482 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e186e0-153d-4030-9024-c91301762a33 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:06.705943 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e186e0-153d-4030-9024-c91301762a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:06.706368 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e186e0-153d-4030-9024-c91301762a33 admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:06.707698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e186e0-153d-4030-9024-c91301762a33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:06.711119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e186e0-153d-4030-9024-c91301762a33 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:06.722807 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1040/2068] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:06 2022] GET /identity/v3/roles => generated 1427 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:56:06.724515 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ab60d-b303-4ec1-9207-ee565b2165e6 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:06.745136 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0ab60d-b303-4ec1-9207-ee565b2165e6 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:06.755764 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88436b-d513-4d87-a41d-45adef7b9a33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:06.756729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88436b-d513-4d87-a41d-45adef7b9a33 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:06.757434 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88436b-d513-4d87-a41d-45adef7b9a33 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:06.758047 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd88436b-d513-4d87-a41d-45adef7b9a33 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:06.759462 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd88436b-d513-4d87-a41d-45adef7b9a33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:06.763809 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd88436b-d513-4d87-a41d-45adef7b9a33 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:06.798233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1041/2069] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:06 2022] GET /identity/v3/roles => generated 1427 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:56:06.807406 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88436b-d513-4d87-a41d-45adef7b9a33 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:06.820279 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd88436b-d513-4d87-a41d-45adef7b9a33 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:06.829646 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1029/2070] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:56:06 2022] POST /identity/v3/users => generated 284 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:56:06.831775 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0231ad4d-0f73-495c-88f1-8e17571cbfd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:06.832818 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231ad4d-0f73-495c-88f1-8e17571cbfd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:06.834022 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e186e0-153d-4030-9024-c91301762a33 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:06.839509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231ad4d-0f73-495c-88f1-8e17571cbfd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:06.839921 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0231ad4d-0f73-495c-88f1-8e17571cbfd6 admin admin] PATH_INFO: `/v3/projects/2e30e47b0d0744d7baa61f43e562ed0a/users/868368a034534937b0c795e929a861bb/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:06.851828 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e186e0-153d-4030-9024-c91301762a33 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:06.860963 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c571215d-cb68-4636-a206-62cdc720e0bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:06.861364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571215d-cb68-4636-a206-62cdc720e0bb admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:06.861646 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571215d-cb68-4636-a206-62cdc720e0bb admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:06.863244 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c571215d-cb68-4636-a206-62cdc720e0bb admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:06.864342 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c571215d-cb68-4636-a206-62cdc720e0bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:06.866218 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c571215d-cb68-4636-a206-62cdc720e0bb admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:06.880629 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0231ad4d-0f73-495c-88f1-8e17571cbfd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e30e47b0d0744d7baa61f43e562ed0a, user_id=868368a034534937b0c795e929a861bb, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:06.884193 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1030/2071] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:06 2022] GET /identity/v3/roles => generated 1427 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:56:06.885077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0231ad4d-0f73-495c-88f1-8e17571cbfd6 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:06.888806 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c571215d-cb68-4636-a206-62cdc720e0bb admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:06.908183 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c571215d-cb68-4636-a206-62cdc720e0bb admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:06.914454 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ab755-771a-44f9-8e66-07d155c178d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:06.915194 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ab755-771a-44f9-8e66-07d155c178d0 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:06.915537 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ab755-771a-44f9-8e66-07d155c178d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:06.915875 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5ab755-771a-44f9-8e66-07d155c178d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:06.916645 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5ab755-771a-44f9-8e66-07d155c178d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:06.919137 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5ab755-771a-44f9-8e66-07d155c178d0 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:06.936027 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1031/2072] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:06 2022] GET /identity/v3/roles => generated 1427 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:56:06.938697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ab755-771a-44f9-8e66-07d155c178d0 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:06.951029 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1042/2073] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:56:06 2022] PUT /identity/v3/projects/2e30e47b0d0744d7baa61f43e562ed0a/users/868368a034534937b0c795e929a861bb/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:06.955246 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5ab755-771a-44f9-8e66-07d155c178d0 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:06.969723 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa196e-bfdd-4cad-983b-976eae949afc None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:06.972623 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa196e-bfdd-4cad-983b-976eae949afc None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:06.976849 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afa196e-bfdd-4cad-983b-976eae949afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:07.028699 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-1afa196e-bfdd-4cad-983b-976eae949afc None None] MFA Rules not processed for user `868368a034534937b0c795e929a861bb`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:07.030434 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-1afa196e-bfdd-4cad-983b-976eae949afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:07.042212 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86626ae-e9cb-4de4-86f0-c4ed5dae8ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:07.044067 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86626ae-e9cb-4de4-86f0-c4ed5dae8ce2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:07.044349 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86626ae-e9cb-4de4-86f0-c4ed5dae8ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:07.044837 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86626ae-e9cb-4de4-86f0-c4ed5dae8ce2 admin admin] PATH_INFO: `/v3/projects/2fd6eae4e5134636b0347a59f76314db/users/5ceb2d06a8074d91ba4de98df2c54fd3/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:07.076811 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86626ae-e9cb-4de4-86f0-c4ed5dae8ce2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2fd6eae4e5134636b0347a59f76314db, user_id=5ceb2d06a8074d91ba4de98df2c54fd3, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:07.080602 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86626ae-e9cb-4de4-86f0-c4ed5dae8ce2 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:07.127537 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1043/2074] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:07.141792 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa196e-bfdd-4cad-983b-976eae949afc None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:07.143593 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-1afa196e-bfdd-4cad-983b-976eae949afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:07.163656 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1032/2075] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:56:06 2022] PUT /identity/v3/projects/2fd6eae4e5134636b0347a59f76314db/users/5ceb2d06a8074d91ba4de98df2c54fd3/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:07.171574 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6534eab-5889-4fa4-9d02-52835aff07cd None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:07.171778 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6534eab-5889-4fa4-9d02-52835aff07cd None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:07.174038 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6534eab-5889-4fa4-9d02-52835aff07cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:07.202957 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-e6534eab-5889-4fa4-9d02-52835aff07cd None None] MFA Rules not processed for user `5ceb2d06a8074d91ba4de98df2c54fd3`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:07.205430 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-e6534eab-5889-4fa4-9d02-52835aff07cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:07.236335 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:07.239599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afa196e-bfdd-4cad-983b-976eae949afc None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:07.285031 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1033/2076] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:07.290724 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6534eab-5889-4fa4-9d02-52835aff07cd None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:07.296077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f80fd0d-879e-48fa-b1e2-9b731dcfda81 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e358fd7c1b3c43d7ac89fb3562678266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:07.297146 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80fd0d-879e-48fa-b1e2-9b731dcfda81 service glance] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:07.297675 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80fd0d-879e-48fa-b1e2-9b731dcfda81 service glance] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:07.298375 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80fd0d-879e-48fa-b1e2-9b731dcfda81 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:07.361680 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:07.375217 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f80fd0d-879e-48fa-b1e2-9b731dcfda81 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:07.376663 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f80fd0d-879e-48fa-b1e2-9b731dcfda81 service glance] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:07.385947 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1044/2077] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:56:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:56:07.386722 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6534eab-5889-4fa4-9d02-52835aff07cd None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:07.395949 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5a401-d8c4-4959-b65a-4ee740cb20a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:07.396603 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5a401-d8c4-4959-b65a-4ee740cb20a2 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:07.396888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5a401-d8c4-4959-b65a-4ee740cb20a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:07.397165 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e5a401-d8c4-4959-b65a-4ee740cb20a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:07.397897 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e5a401-d8c4-4959-b65a-4ee740cb20a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:07.399878 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e5a401-d8c4-4959-b65a-4ee740cb20a2 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:07.519089 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1034/2078] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:56:07 2022] POST /identity/v3/projects => generated 364 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:56:07.524855 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f80fd0d-879e-48fa-b1e2-9b731dcfda81 service glance] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:07.547357 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f80fd0d-879e-48fa-b1e2-9b731dcfda81 service glance] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:07.615922 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b8ca1-631e-4bd6-b428-e52b42c44b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:07.616580 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b8ca1-631e-4bd6-b428-e52b42c44b7c admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:07.617074 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b8ca1-631e-4bd6-b428-e52b42c44b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:07.617192 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728b8ca1-631e-4bd6-b428-e52b42c44b7c admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:07.617797 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b8ca1-631e-4bd6-b428-e52b42c44b7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:07.619795 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728b8ca1-631e-4bd6-b428-e52b42c44b7c admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:07.726075 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1045/2079] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:56:07 2022] POST /identity/v3/users => generated 289 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:56:07.731222 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5a401-d8c4-4959-b65a-4ee740cb20a2 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:07.751487 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e5a401-d8c4-4959-b65a-4ee740cb20a2 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:07.760966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95efac-2ade-48b1-8e3c-51bee6f5812f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:07.761737 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95efac-2ade-48b1-8e3c-51bee6f5812f admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:07.762641 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95efac-2ade-48b1-8e3c-51bee6f5812f admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:07.764392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e95efac-2ade-48b1-8e3c-51bee6f5812f admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:07.764392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e95efac-2ade-48b1-8e3c-51bee6f5812f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:07.765594 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e95efac-2ade-48b1-8e3c-51bee6f5812f admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:07.776505 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1035/2080] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:07 2022] GET /identity/v3/roles => generated 1427 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:56:07.781918 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b8ca1-631e-4bd6-b428-e52b42c44b7c admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:07.792251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728b8ca1-631e-4bd6-b428-e52b42c44b7c admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:07.798158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c825c4-1c45-4fb1-a30d-968622ea8af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:07.798828 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c825c4-1c45-4fb1-a30d-968622ea8af7 admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:07.799135 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c825c4-1c45-4fb1-a30d-968622ea8af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:07.799410 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c825c4-1c45-4fb1-a30d-968622ea8af7 admin admin] PATH_INFO: `/v3/projects/195f38310fb24f40be5b4a346607018d/users/be88c74ac3be4e4780e82df578aa7c43/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:07.817446 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c825c4-1c45-4fb1-a30d-968622ea8af7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=195f38310fb24f40be5b4a346607018d, user_id=be88c74ac3be4e4780e82df578aa7c43, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:07.821757 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c825c4-1c45-4fb1-a30d-968622ea8af7 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:07.920482 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1046/2081] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:56:07 2022] PUT /identity/v3/projects/195f38310fb24f40be5b4a346607018d/users/be88c74ac3be4e4780e82df578aa7c43/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:56:07.932334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95efac-2ade-48b1-8e3c-51bee6f5812f admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:07.948216 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e95efac-2ade-48b1-8e3c-51bee6f5812f admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:07.970007 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c825c4-1c45-4fb1-a30d-968622ea8af7 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:07.970007 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-08c825c4-1c45-4fb1-a30d-968622ea8af7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:08.041339 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c825c4-1c45-4fb1-a30d-968622ea8af7 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:08.051926 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4587184-7e0b-4f4e-9af8-0234321768c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:08.061585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4587184-7e0b-4f4e-9af8-0234321768c2 service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:08.061585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4587184-7e0b-4f4e-9af8-0234321768c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:08.061585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4587184-7e0b-4f4e-9af8-0234321768c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:08.061585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-f4587184-7e0b-4f4e-9af8-0234321768c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:08.062336 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ceecab-c4aa-4d22-9d58-9ef56fdef361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:08.062336 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ceecab-c4aa-4d22-9d58-9ef56fdef361 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:08.062336 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ceecab-c4aa-4d22-9d58-9ef56fdef361 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:08.062336 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ceecab-c4aa-4d22-9d58-9ef56fdef361 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:08.062673 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ceecab-c4aa-4d22-9d58-9ef56fdef361 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:08.062673 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ceecab-c4aa-4d22-9d58-9ef56fdef361 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:08.077463 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1036/2082] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:07 2022] GET /identity/v3/roles => generated 1427 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:56:08.083449 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:08.085066 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ceecab-c4aa-4d22-9d58-9ef56fdef361 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:08.097277 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ceecab-c4aa-4d22-9d58-9ef56fdef361 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:08.108162 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8a6dae-5a50-4d51-a62e-d169ed82bcbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:08.113216 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a6dae-5a50-4d51-a62e-d169ed82bcbd admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:08.113216 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a6dae-5a50-4d51-a62e-d169ed82bcbd admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:08.113216 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8a6dae-5a50-4d51-a62e-d169ed82bcbd admin admin] PATH_INFO: `/v3/projects/195f38310fb24f40be5b4a346607018d/users/be88c74ac3be4e4780e82df578aa7c43/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:08.116789 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8a6dae-5a50-4d51-a62e-d169ed82bcbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=195f38310fb24f40be5b4a346607018d, user_id=be88c74ac3be4e4780e82df578aa7c43, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:08.119996 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8a6dae-5a50-4d51-a62e-d169ed82bcbd admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:08.133813 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1037/2083] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:56:08 2022] PUT /identity/v3/projects/195f38310fb24f40be5b4a346607018d/users/be88c74ac3be4e4780e82df578aa7c43/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:08.146789 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720976b9-fe05-41b8-9e7c-dbd721589858 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:08.147085 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720976b9-fe05-41b8-9e7c-dbd721589858 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:08.147418 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720976b9-fe05-41b8-9e7c-dbd721589858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:08.151291 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4587184-7e0b-4f4e-9af8-0234321768c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:08.153816 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4587184-7e0b-4f4e-9af8-0234321768c2 service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:08.171293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1047/2084] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:56:07 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:56:08.184365 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-720976b9-fe05-41b8-9e7c-dbd721589858 None None] MFA Rules not processed for user `be88c74ac3be4e4780e82df578aa7c43`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:08.185708 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-720976b9-fe05-41b8-9e7c-dbd721589858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:08.272184 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1038/2085] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:08 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:08.319672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:08.380398 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4587184-7e0b-4f4e-9af8-0234321768c2 service cinder] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:08.394554 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720976b9-fe05-41b8-9e7c-dbd721589858 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:08.395932 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4587184-7e0b-4f4e-9af8-0234321768c2 service cinder] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:08.397139 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-720976b9-fe05-41b8-9e7c-dbd721589858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:08.510570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5084a3f-b090-46de-994a-05a605f8af84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:08.510867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5084a3f-b090-46de-994a-05a605f8af84 service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:08.511351 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5084a3f-b090-46de-994a-05a605f8af84 service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:08.511602 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720976b9-fe05-41b8-9e7c-dbd721589858 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:08.512161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5084a3f-b090-46de-994a-05a605f8af84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:08.516640 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a90ddb-c755-40cf-bb4a-c15c83411e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:08.517286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a90ddb-c755-40cf-bb4a-c15c83411e31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:08.517589 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a90ddb-c755-40cf-bb4a-c15c83411e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:08.517865 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a90ddb-c755-40cf-bb4a-c15c83411e31 admin admin] PATH_INFO: `/v3/users/5e0379bb8134440aa57ff1e89f068a60` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:08.519089 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a90ddb-c755-40cf-bb4a-c15c83411e31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e0379bb8134440aa57ff1e89f068a60)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:08.521029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a90ddb-c755-40cf-bb4a-c15c83411e31 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:08.524936 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5084a3f-b090-46de-994a-05a605f8af84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:08.526393 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5084a3f-b090-46de-994a-05a605f8af84 service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:08.540788 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:08.551461 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1048/2086] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:56:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:08.691458 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:08.847574 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-52a90ddb-c755-40cf-bb4a-c15c83411e31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e0379bb8134440aa57ff1e89f068a60'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:08.857010 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5084a3f-b090-46de-994a-05a605f8af84 service cinder] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:08.868928 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5084a3f-b090-46de-994a-05a605f8af84 service cinder] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:08.908874 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fad12db-9343-4cd4-9b94-c40f16791afc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:08.909368 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-52a90ddb-c755-40cf-bb4a-c15c83411e31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e0379bb8134440aa57ff1e89f068a60'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:08.909510 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad12db-9343-4cd4-9b94-c40f16791afc service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:08.909793 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad12db-9343-4cd4-9b94-c40f16791afc service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:08.910068 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fad12db-9343-4cd4-9b94-c40f16791afc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:08.910772 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-52a90ddb-c755-40cf-bb4a-c15c83411e31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e0379bb8134440aa57ff1e89f068a60'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:09.001693 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-52a90ddb-c755-40cf-bb4a-c15c83411e31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e0379bb8134440aa57ff1e89f068a60'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:09.003707 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad12db-9343-4cd4-9b94-c40f16791afc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:09.007507 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fad12db-9343-4cd4-9b94-c40f16791afc service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:09.024560 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-52a90ddb-c755-40cf-bb4a-c15c83411e31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e0379bb8134440aa57ff1e89f068a60'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:09.026925 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1049/2087] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:56:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:09.040871 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1039/2088] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:56:08 2022] DELETE /identity/v3/users/5e0379bb8134440aa57ff1e89f068a60 => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:09.045895 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc84f0d-a152-4805-a3fc-8de5cdab59c9 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:09.047462 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc84f0d-a152-4805-a3fc-8de5cdab59c9 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:09.047980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc84f0d-a152-4805-a3fc-8de5cdab59c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:09.096314 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-fbc84f0d-a152-4805-a3fc-8de5cdab59c9 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:09.096314 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-fbc84f0d-a152-4805-a3fc-8de5cdab59c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:09.161884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:09.179369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1050/2089] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:09 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:09.187374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a90ddb-c755-40cf-bb4a-c15c83411e31 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:09.193882 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-52a90ddb-c755-40cf-bb4a-c15c83411e31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:09.306720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a90ddb-c755-40cf-bb4a-c15c83411e31 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:09.314294 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c661c6-f5d0-4367-92de-6e3777692781 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:09.315127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c661c6-f5d0-4367-92de-6e3777692781 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:09.315448 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c661c6-f5d0-4367-92de-6e3777692781 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:09.315928 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c661c6-f5d0-4367-92de-6e3777692781 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:09.339890 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c661c6-f5d0-4367-92de-6e3777692781 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:09.341466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c661c6-f5d0-4367-92de-6e3777692781 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:09.360902 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1040/2090] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:56:09 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:10.049411 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc84f0d-a152-4805-a3fc-8de5cdab59c9 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:10.050992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-fbc84f0d-a152-4805-a3fc-8de5cdab59c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:10.065802 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc84f0d-a152-4805-a3fc-8de5cdab59c9 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:10.077896 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b184a-b581-4a63-bf4a-1a0204f4719e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:10.078651 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b184a-b581-4a63-bf4a-1a0204f4719e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:10.079291 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b184a-b581-4a63-bf4a-1a0204f4719e admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:10.079777 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b184a-b581-4a63-bf4a-1a0204f4719e admin admin] PATH_INFO: `/v3/projects/6e8a725726a549cfa6873fa0571062c2` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:10.081220 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b184a-b581-4a63-bf4a-1a0204f4719e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e8a725726a549cfa6873fa0571062c2)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:10.084770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b184a-b581-4a63-bf4a-1a0204f4719e admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:10.232504 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-df7b184a-b581-4a63-bf4a-1a0204f4719e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e8a725726a549cfa6873fa0571062c2', 'request_id': 'req-df7b184a-b581-4a63-bf4a-1a0204f4719e'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:10.232504 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-df7b184a-b581-4a63-bf4a-1a0204f4719e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e8a725726a549cfa6873fa0571062c2', 'request_id': 'req-df7b184a-b581-4a63-bf4a-1a0204f4719e'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:10.331878 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-df7b184a-b581-4a63-bf4a-1a0204f4719e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e8a725726a549cfa6873fa0571062c2', 'request_id': 'req-df7b184a-b581-4a63-bf4a-1a0204f4719e'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:10.348387 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-df7b184a-b581-4a63-bf4a-1a0204f4719e admin admin] The token cache is being invalidate because project 6e8a725726a549cfa6873fa0571062c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:56:10.349381 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-df7b184a-b581-4a63-bf4a-1a0204f4719e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:10.350471 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-df7b184a-b581-4a63-bf4a-1a0204f4719e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:10.352498 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1051/2091] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:56:10 2022] DELETE /identity/v3/projects/6e8a725726a549cfa6873fa0571062c2 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:10.357887 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b53c4d-09f4-4f3e-be22-4ba8dae22558 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:10.358443 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b53c4d-09f4-4f3e-be22-4ba8dae22558 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:10.358613 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b53c4d-09f4-4f3e-be22-4ba8dae22558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:10.391256 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-d3b53c4d-09f4-4f3e-be22-4ba8dae22558 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:10.392478 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-d3b53c4d-09f4-4f3e-be22-4ba8dae22558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:10.447742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:10.486230 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1041/2092] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:10.493956 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c73191-ccc0-4170-9de9-c542a538812c None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:10.494429 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c73191-ccc0-4170-9de9-c542a538812c None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:10.494808 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c73191-ccc0-4170-9de9-c542a538812c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:10.521105 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-60c73191-ccc0-4170-9de9-c542a538812c None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:10.522338 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-60c73191-ccc0-4170-9de9-c542a538812c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:10.531632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1052/2093] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:10.535754 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b53c4d-09f4-4f3e-be22-4ba8dae22558 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:10.546863 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b53c4d-09f4-4f3e-be22-4ba8dae22558 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:10.551781 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767b90a-ea9a-4edf-bd00-5feb405610a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:10.552404 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767b90a-ea9a-4edf-bd00-5feb405610a3 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:10.552738 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767b90a-ea9a-4edf-bd00-5feb405610a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:10.553134 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c767b90a-ea9a-4edf-bd00-5feb405610a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:10.553854 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c767b90a-ea9a-4edf-bd00-5feb405610a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:56:10.554457 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c767b90a-ea9a-4edf-bd00-5feb405610a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:10.555784 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c767b90a-ea9a-4edf-bd00-5feb405610a3 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:10.587428 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1042/2094] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:56:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:56:10.587512 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c73191-ccc0-4170-9de9-c542a538812c None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:10.622402 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c73191-ccc0-4170-9de9-c542a538812c None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:10.628221 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a42d97-3a70-4d25-8be7-a7fb33e99c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:10.628950 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a42d97-3a70-4d25-8be7-a7fb33e99c38 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:10.629425 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a42d97-3a70-4d25-8be7-a7fb33e99c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:10.629761 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a42d97-3a70-4d25-8be7-a7fb33e99c38 admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:10.630508 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a42d97-3a70-4d25-8be7-a7fb33e99c38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:10.645612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a42d97-3a70-4d25-8be7-a7fb33e99c38 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:10.831247 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1053/2095] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:56:10 2022] POST /identity/v3/projects => generated 376 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:56:10.836577 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767b90a-ea9a-4edf-bd00-5feb405610a3 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:10.857685 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c767b90a-ea9a-4edf-bd00-5feb405610a3 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:10.950272 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac84d3-d906-4615-9ece-b0ec1b9a4155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:10.950978 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac84d3-d906-4615-9ece-b0ec1b9a4155 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:10.951652 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac84d3-d906-4615-9ece-b0ec1b9a4155 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:10.952274 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abac84d3-d906-4615-9ece-b0ec1b9a4155 admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:10.956295 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac84d3-d906-4615-9ece-b0ec1b9a4155 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:10.958515 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abac84d3-d906-4615-9ece-b0ec1b9a4155 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:11.104117 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1043/2096] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:56:10 2022] POST /identity/v3/users => generated 289 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:56:11.114409 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a42d97-3a70-4d25-8be7-a7fb33e99c38 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:11.131069 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a42d97-3a70-4d25-8be7-a7fb33e99c38 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:11.140331 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa5aa8-5b69-45e8-8bf0-f633ef888df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:11.140331 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa5aa8-5b69-45e8-8bf0-f633ef888df0 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:11.140331 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa5aa8-5b69-45e8-8bf0-f633ef888df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:11.140331 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9faa5aa8-5b69-45e8-8bf0-f633ef888df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:11.140678 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faa5aa8-5b69-45e8-8bf0-f633ef888df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:11.141695 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9faa5aa8-5b69-45e8-8bf0-f633ef888df0 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:11.154771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1054/2097] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:11 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:56:11.159396 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac84d3-d906-4615-9ece-b0ec1b9a4155 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:11.172582 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abac84d3-d906-4615-9ece-b0ec1b9a4155 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:11.179485 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef50e6-978e-4ac8-8bde-baadd90391bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:11.181215 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef50e6-978e-4ac8-8bde-baadd90391bd admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:11.181215 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef50e6-978e-4ac8-8bde-baadd90391bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:11.181549 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef50e6-978e-4ac8-8bde-baadd90391bd admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:11.182503 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef50e6-978e-4ac8-8bde-baadd90391bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:11.185267 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef50e6-978e-4ac8-8bde-baadd90391bd admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:11.207001 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1044/2098] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:11 2022] GET /identity/v3/roles => generated 1427 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:56:11.208860 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa5aa8-5b69-45e8-8bf0-f633ef888df0 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:11.220818 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9faa5aa8-5b69-45e8-8bf0-f633ef888df0 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:11.226331 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f46e507-8418-408e-9c0f-dd4c1554f136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:11.227016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f46e507-8418-408e-9c0f-dd4c1554f136 admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:11.227386 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f46e507-8418-408e-9c0f-dd4c1554f136 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:11.227677 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f46e507-8418-408e-9c0f-dd4c1554f136 admin admin] PATH_INFO: `/v3/projects/9a19b82d3ae74f0c9bd89c9cbf6533d6/users/4d1510bdacfb438a8b1118f9a0776b29/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:11.255916 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f46e507-8418-408e-9c0f-dd4c1554f136 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a19b82d3ae74f0c9bd89c9cbf6533d6, user_id=4d1510bdacfb438a8b1118f9a0776b29, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:11.260919 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f46e507-8418-408e-9c0f-dd4c1554f136 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:11.356530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1055/2099] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:56:11 2022] PUT /identity/v3/projects/9a19b82d3ae74f0c9bd89c9cbf6533d6/users/4d1510bdacfb438a8b1118f9a0776b29/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:11.367019 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c2f46-8b73-4e3d-b137-3e54087beea1 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:11.367543 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c2f46-8b73-4e3d-b137-3e54087beea1 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:11.367638 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c2f46-8b73-4e3d-b137-3e54087beea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:11.408210 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-a14c2f46-8b73-4e3d-b137-3e54087beea1 None None] MFA Rules not processed for user `4d1510bdacfb438a8b1118f9a0776b29`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:11.409713 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-a14c2f46-8b73-4e3d-b137-3e54087beea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:11.476780 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0073c8-24b8-4433-9a21-692ad3a095f3 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:11.477093 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0073c8-24b8-4433-9a21-692ad3a095f3 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:11.477617 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0073c8-24b8-4433-9a21-692ad3a095f3 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:11.478917 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1056/2100] 10.222.0.115 () {54 vars in 926 bytes} [Tue Aug 2 10:56:11 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:56:11.503071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1045/2101] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:11.515071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0073c8-24b8-4433-9a21-692ad3a095f3 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:11.517025 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec810bb3-b563-4166-ae96-99323d100495 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:11.517175 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-9d0073c8-24b8-4433-9a21-692ad3a095f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:11.517606 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec810bb3-b563-4166-ae96-99323d100495 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:11.517885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec810bb3-b563-4166-ae96-99323d100495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:11.539181 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-ec810bb3-b563-4166-ae96-99323d100495 None None] MFA Rules not processed for user `cbc4b962e2a742b08beef7187cc527e5`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:11.540502 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-ec810bb3-b563-4166-ae96-99323d100495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:11.587792 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0073c8-24b8-4433-9a21-692ad3a095f3 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:11.592719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:11.595600 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8387aad5-097a-40f8-9fe2-55035f8c7d35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:11.596242 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387aad5-097a-40f8-9fe2-55035f8c7d35 service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:11.596526 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387aad5-097a-40f8-9fe2-55035f8c7d35 service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:11.596801 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387aad5-097a-40f8-9fe2-55035f8c7d35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:11.626555 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8387aad5-097a-40f8-9fe2-55035f8c7d35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:11.628380 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8387aad5-097a-40f8-9fe2-55035f8c7d35 service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:11.636286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1046/2102] 10.222.0.115 () {58 vars in 1023 bytes} [Tue Aug 2 10:56:11 2022] POST /identity/v3/auth/tokens => generated 3318 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:56:11.659608 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec810bb3-b563-4166-ae96-99323d100495 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:11.659608 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-ec810bb3-b563-4166-ae96-99323d100495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:11.661043 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1057/2103] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:56:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:56:11.675199 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec810bb3-b563-4166-ae96-99323d100495 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:11.684874 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0c539-6f22-402f-812a-2e8b58da6de1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc4b962e2a742b08beef7187cc527e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:11.685986 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0c539-6f22-402f-812a-2e8b58da6de1 service placement] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:11.689658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0c539-6f22-402f-812a-2e8b58da6de1 service placement] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:11.689658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b0c539-6f22-402f-812a-2e8b58da6de1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:11.691085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8387aad5-097a-40f8-9fe2-55035f8c7d35 service cinder] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:11.692584 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-8387aad5-097a-40f8-9fe2-55035f8c7d35 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:11.706668 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0c539-6f22-402f-812a-2e8b58da6de1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:11.708490 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b0c539-6f22-402f-812a-2e8b58da6de1 service placement] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:11.728257 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1047/2104] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:56:11 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:11.737837 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:11.797595 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8387aad5-097a-40f8-9fe2-55035f8c7d35 service cinder] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:11.803761 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34151046-1286-44cf-9a28-72c9692378a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:11.804421 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34151046-1286-44cf-9a28-72c9692378a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:11.804708 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34151046-1286-44cf-9a28-72c9692378a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:11.804987 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34151046-1286-44cf-9a28-72c9692378a9 admin admin] PATH_INFO: `/v3/users/822e6ac5ec8347c9815c770738509d30` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:11.806242 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34151046-1286-44cf-9a28-72c9692378a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=822e6ac5ec8347c9815c770738509d30)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:11.808311 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34151046-1286-44cf-9a28-72c9692378a9 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:12.062657 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-34151046-1286-44cf-9a28-72c9692378a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '822e6ac5ec8347c9815c770738509d30'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:12.063289 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-34151046-1286-44cf-9a28-72c9692378a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '822e6ac5ec8347c9815c770738509d30'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:12.133927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-34151046-1286-44cf-9a28-72c9692378a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '822e6ac5ec8347c9815c770738509d30'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:12.144909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-34151046-1286-44cf-9a28-72c9692378a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '822e6ac5ec8347c9815c770738509d30'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:12.163089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-34151046-1286-44cf-9a28-72c9692378a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '822e6ac5ec8347c9815c770738509d30'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:12.169748 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1058/2105] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:56:11 2022] DELETE /identity/v3/users/822e6ac5ec8347c9815c770738509d30 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:56:12.428548 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0c539-6f22-402f-812a-2e8b58da6de1 service placement] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:12.450436 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b0c539-6f22-402f-812a-2e8b58da6de1 service placement] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:12.526067 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0755bff8-7f25-4076-bb4b-06d40d5d4116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:12.526744 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0755bff8-7f25-4076-bb4b-06d40d5d4116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:12.527018 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0755bff8-7f25-4076-bb4b-06d40d5d4116 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:12.527826 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0755bff8-7f25-4076-bb4b-06d40d5d4116 admin admin] PATH_INFO: `/v3/projects/5ced4a8a26d94c10b08bbca381d62d34` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:12.529273 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0755bff8-7f25-4076-bb4b-06d40d5d4116 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ced4a8a26d94c10b08bbca381d62d34)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:12.531432 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0755bff8-7f25-4076-bb4b-06d40d5d4116 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:12.714615 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-0755bff8-7f25-4076-bb4b-06d40d5d4116 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ced4a8a26d94c10b08bbca381d62d34', 'request_id': 'req-0755bff8-7f25-4076-bb4b-06d40d5d4116'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:12.784199 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-0755bff8-7f25-4076-bb4b-06d40d5d4116 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ced4a8a26d94c10b08bbca381d62d34', 'request_id': 'req-0755bff8-7f25-4076-bb4b-06d40d5d4116'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:12.792436 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-0755bff8-7f25-4076-bb4b-06d40d5d4116 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ced4a8a26d94c10b08bbca381d62d34', 'request_id': 'req-0755bff8-7f25-4076-bb4b-06d40d5d4116'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:12.804789 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-0755bff8-7f25-4076-bb4b-06d40d5d4116 admin admin] The token cache is being invalidate because project 5ced4a8a26d94c10b08bbca381d62d34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:56:12.805137 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-0755bff8-7f25-4076-bb4b-06d40d5d4116 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:12.805865 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-0755bff8-7f25-4076-bb4b-06d40d5d4116 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:12.807455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1048/2106] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:56:12 2022] DELETE /identity/v3/projects/5ced4a8a26d94c10b08bbca381d62d34 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:12.812543 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40014dff-8eb9-4682-bf1d-f00ffd896601 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:12.812838 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40014dff-8eb9-4682-bf1d-f00ffd896601 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:12.813161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40014dff-8eb9-4682-bf1d-f00ffd896601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:12.834065 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-40014dff-8eb9-4682-bf1d-f00ffd896601 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:12.835312 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-40014dff-8eb9-4682-bf1d-f00ffd896601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:12.895932 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1059/2107] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:12 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:12.907123 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43df617-9efe-4d20-810e-a6723cfddaab None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:12.913836 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43df617-9efe-4d20-810e-a6723cfddaab None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:12.913836 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43df617-9efe-4d20-810e-a6723cfddaab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:12.935329 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40014dff-8eb9-4682-bf1d-f00ffd896601 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:12.936569 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-f43df617-9efe-4d20-810e-a6723cfddaab None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:12.936901 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-40014dff-8eb9-4682-bf1d-f00ffd896601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:12.938092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f43df617-9efe-4d20-810e-a6723cfddaab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:12.946339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1049/2108] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:12 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:12.949731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40014dff-8eb9-4682-bf1d-f00ffd896601 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:12.951480 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43df617-9efe-4d20-810e-a6723cfddaab None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:12.954980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02274f9-d936-458c-acc2-5a3c2d3177fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:12.955653 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02274f9-d936-458c-acc2-5a3c2d3177fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:12.955943 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02274f9-d936-458c-acc2-5a3c2d3177fa admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:12.956209 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02274f9-d936-458c-acc2-5a3c2d3177fa admin admin] PATH_INFO: `/v3/users/5ceb2d06a8074d91ba4de98df2c54fd3` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:12.957386 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02274f9-d936-458c-acc2-5a3c2d3177fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ceb2d06a8074d91ba4de98df2c54fd3)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:12.959528 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02274f9-d936-458c-acc2-5a3c2d3177fa admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:12.961411 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43df617-9efe-4d20-810e-a6723cfddaab None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:12.966972 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6812e066-9b5c-47ad-b944-9fbe24006fa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:12.967617 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812e066-9b5c-47ad-b944-9fbe24006fa7 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:12.967925 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812e066-9b5c-47ad-b944-9fbe24006fa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:12.968200 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812e066-9b5c-47ad-b944-9fbe24006fa7 admin admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:12.968898 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6812e066-9b5c-47ad-b944-9fbe24006fa7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:56:12.969203 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6812e066-9b5c-47ad-b944-9fbe24006fa7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:12.970802 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6812e066-9b5c-47ad-b944-9fbe24006fa7 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:12.983674 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1050/2109] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:56:12 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:56:12.988873 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6812e066-9b5c-47ad-b944-9fbe24006fa7 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:12.998942 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6812e066-9b5c-47ad-b944-9fbe24006fa7 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:13.003660 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd99415-88a9-4ce6-b684-f70c6e1dcdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:13.005477 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd99415-88a9-4ce6-b684-f70c6e1dcdff admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:13.005477 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd99415-88a9-4ce6-b684-f70c6e1dcdff admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:13.005477 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd99415-88a9-4ce6-b684-f70c6e1dcdff admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:13.005477 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd99415-88a9-4ce6-b684-f70c6e1dcdff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:13.007305 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd99415-88a9-4ce6-b684-f70c6e1dcdff admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:13.123028 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1051/2110] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:56:12 2022] POST /identity/v3/projects => generated 394 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:56:13.127379 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd99415-88a9-4ce6-b684-f70c6e1dcdff admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:13.138260 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:13.143891 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:13.158360 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c02274f9-d936-458c-acc2-5a3c2d3177fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ceb2d06a8074d91ba4de98df2c54fd3'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:13.159023 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c02274f9-d936-458c-acc2-5a3c2d3177fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ceb2d06a8074d91ba4de98df2c54fd3'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:13.193409 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd99415-88a9-4ce6-b684-f70c6e1dcdff admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:13.198086 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec8ca1-fad1-421c-a240-132428f3c1ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:13.198711 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec8ca1-fad1-421c-a240-132428f3c1ff admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:13.198990 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec8ca1-fad1-421c-a240-132428f3c1ff admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:13.199287 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec8ca1-fad1-421c-a240-132428f3c1ff admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:13.200938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec8ca1-fad1-421c-a240-132428f3c1ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:13.203008 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eec8ca1-fad1-421c-a240-132428f3c1ff admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:13.258737 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c02274f9-d936-458c-acc2-5a3c2d3177fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ceb2d06a8074d91ba4de98df2c54fd3'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:13.270301 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c02274f9-d936-458c-acc2-5a3c2d3177fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ceb2d06a8074d91ba4de98df2c54fd3'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:13.283738 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c02274f9-d936-458c-acc2-5a3c2d3177fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ceb2d06a8074d91ba4de98df2c54fd3'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:13.291535 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1060/2111] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:56:12 2022] DELETE /identity/v3/users/5ceb2d06a8074d91ba4de98df2c54fd3 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:56:13.306621 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d366de-5f09-4789-8d17-40f27dd6f786 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:13.306621 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d366de-5f09-4789-8d17-40f27dd6f786 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:13.306621 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d366de-5f09-4789-8d17-40f27dd6f786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:13.316980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-e6d366de-5f09-4789-8d17-40f27dd6f786 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:13.318343 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-e6d366de-5f09-4789-8d17-40f27dd6f786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:13.337617 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1052/2112] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:56:13 2022] POST /identity/v3/users => generated 298 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:56:13.343988 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec8ca1-fad1-421c-a240-132428f3c1ff admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:13.345946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-3eec8ca1-fad1-421c-a240-132428f3c1ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:13.359134 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eec8ca1-fad1-421c-a240-132428f3c1ff admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:13.386168 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1061/2113] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:13.398357 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d366de-5f09-4789-8d17-40f27dd6f786 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:13.401010 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-e6d366de-5f09-4789-8d17-40f27dd6f786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:13.444768 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a0e8c-511f-425f-8776-d889019b7c9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:13.444768 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a0e8c-511f-425f-8776-d889019b7c9a admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:13.444768 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a0e8c-511f-425f-8776-d889019b7c9a admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:13.444768 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a0e8c-511f-425f-8776-d889019b7c9a admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:13.445570 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a0e8c-511f-425f-8776-d889019b7c9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:13.445570 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a0e8c-511f-425f-8776-d889019b7c9a admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:13.445570 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1053/2114] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:13 2022] GET /identity/v3/roles => generated 1427 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:56:13.448677 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a0e8c-511f-425f-8776-d889019b7c9a admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:13.459241 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a0e8c-511f-425f-8776-d889019b7c9a admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:13.463960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef80df3-50b5-455b-918d-3e8de24d6e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:13.464585 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef80df3-50b5-455b-918d-3e8de24d6e36 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:13.464945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef80df3-50b5-455b-918d-3e8de24d6e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:13.465225 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef80df3-50b5-455b-918d-3e8de24d6e36 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:13.466391 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef80df3-50b5-455b-918d-3e8de24d6e36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:13.468099 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef80df3-50b5-455b-918d-3e8de24d6e36 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:13.469940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d366de-5f09-4789-8d17-40f27dd6f786 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:13.473820 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d4ba5-3e2c-4b1e-bb83-76c15860faf1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:13.474467 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d4ba5-3e2c-4b1e-bb83-76c15860faf1 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:13.474743 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d4ba5-3e2c-4b1e-bb83-76c15860faf1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:13.478624 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d4ba5-3e2c-4b1e-bb83-76c15860faf1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:13.479290 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1054/2115] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:13 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:56:13.484540 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef80df3-50b5-455b-918d-3e8de24d6e36 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:13.485764 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d4ba5-3e2c-4b1e-bb83-76c15860faf1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:13.487478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d4ba5-3e2c-4b1e-bb83-76c15860faf1 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:13.495783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef80df3-50b5-455b-918d-3e8de24d6e36 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:13.504415 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b232b-775a-4ff4-affd-9356017af352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:13.505201 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b232b-775a-4ff4-affd-9356017af352 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:13.506872 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b232b-775a-4ff4-affd-9356017af352 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:13.506872 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613b232b-775a-4ff4-affd-9356017af352 admin admin] PATH_INFO: `/v3/projects/0d99d3fd13df429a8ee8a7cb7780adb5/users/3057a53debc34c829372c03567e1119d/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:13.514166 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1062/2116] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:56:13 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:13.523953 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b232b-775a-4ff4-affd-9356017af352 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d99d3fd13df429a8ee8a7cb7780adb5, user_id=3057a53debc34c829372c03567e1119d, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:13.527084 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613b232b-775a-4ff4-affd-9356017af352 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:13.647693 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1055/2117] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:56:13 2022] PUT /identity/v3/projects/0d99d3fd13df429a8ee8a7cb7780adb5/users/3057a53debc34c829372c03567e1119d/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:56:13.653309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5372dc-246e-4b5e-b951-7645f2b067a8 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:13.653309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5372dc-246e-4b5e-b951-7645f2b067a8 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:13.653655 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5372dc-246e-4b5e-b951-7645f2b067a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:13.675350 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-3f5372dc-246e-4b5e-b951-7645f2b067a8 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:13.676793 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-3f5372dc-246e-4b5e-b951-7645f2b067a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:13.742883 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:13.767254 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1063/2118] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:13 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:13.785522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b232b-775a-4ff4-affd-9356017af352 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:13.855516 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613b232b-775a-4ff4-affd-9356017af352 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:13.860954 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824fb40e-e7a2-4f53-a4d0-af024a49a736 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:13.861829 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824fb40e-e7a2-4f53-a4d0-af024a49a736 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:13.862114 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824fb40e-e7a2-4f53-a4d0-af024a49a736 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:13.862519 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824fb40e-e7a2-4f53-a4d0-af024a49a736 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:13.876255 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824fb40e-e7a2-4f53-a4d0-af024a49a736 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:13.877872 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824fb40e-e7a2-4f53-a4d0-af024a49a736 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:13.900423 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1056/2119] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:56:13 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:14.082653 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5372dc-246e-4b5e-b951-7645f2b067a8 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:14.084906 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-3f5372dc-246e-4b5e-b951-7645f2b067a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:14.100411 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5372dc-246e-4b5e-b951-7645f2b067a8 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:14.110620 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce32c65-4fd0-4388-88f4-f4ed0763e34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:14.111437 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce32c65-4fd0-4388-88f4-f4ed0763e34d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:14.117001 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce32c65-4fd0-4388-88f4-f4ed0763e34d admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:14.117001 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce32c65-4fd0-4388-88f4-f4ed0763e34d admin admin] PATH_INFO: `/v3/projects/2fd6eae4e5134636b0347a59f76314db` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:14.117001 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce32c65-4fd0-4388-88f4-f4ed0763e34d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2fd6eae4e5134636b0347a59f76314db)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:14.117374 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce32c65-4fd0-4388-88f4-f4ed0763e34d admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:14.281752 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-8ce32c65-4fd0-4388-88f4-f4ed0763e34d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2fd6eae4e5134636b0347a59f76314db', 'request_id': 'req-8ce32c65-4fd0-4388-88f4-f4ed0763e34d'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:14.306192 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-8ce32c65-4fd0-4388-88f4-f4ed0763e34d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2fd6eae4e5134636b0347a59f76314db', 'request_id': 'req-8ce32c65-4fd0-4388-88f4-f4ed0763e34d'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:14.411337 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-8ce32c65-4fd0-4388-88f4-f4ed0763e34d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2fd6eae4e5134636b0347a59f76314db', 'request_id': 'req-8ce32c65-4fd0-4388-88f4-f4ed0763e34d'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:14.423945 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-8ce32c65-4fd0-4388-88f4-f4ed0763e34d admin admin] The token cache is being invalidate because project 2fd6eae4e5134636b0347a59f76314db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:56:14.426218 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-8ce32c65-4fd0-4388-88f4-f4ed0763e34d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:14.426218 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-8ce32c65-4fd0-4388-88f4-f4ed0763e34d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:14.426661 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1064/2120] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:56:14 2022] DELETE /identity/v3/projects/2fd6eae4e5134636b0347a59f76314db => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:14.430800 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824fb40e-e7a2-4f53-a4d0-af024a49a736 service neutron] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:14.436968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:14.443040 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824fb40e-e7a2-4f53-a4d0-af024a49a736 service neutron] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:14.506058 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef23b2c1-4e66-40ec-b45c-1186ba98f90b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:14.506776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23b2c1-4e66-40ec-b45c-1186ba98f90b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:14.507081 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23b2c1-4e66-40ec-b45c-1186ba98f90b admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:14.511395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef23b2c1-4e66-40ec-b45c-1186ba98f90b admin admin] PATH_INFO: `/v3/users/be88c74ac3be4e4780e82df578aa7c43` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:14.511395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef23b2c1-4e66-40ec-b45c-1186ba98f90b admin admin] RBAC: Authorizing `identity:delete_user(user_id=be88c74ac3be4e4780e82df578aa7c43)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:14.511395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef23b2c1-4e66-40ec-b45c-1186ba98f90b admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:14.832430 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ef23b2c1-4e66-40ec-b45c-1186ba98f90b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be88c74ac3be4e4780e82df578aa7c43'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:14.839223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ef23b2c1-4e66-40ec-b45c-1186ba98f90b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be88c74ac3be4e4780e82df578aa7c43'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:14.839916 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ef23b2c1-4e66-40ec-b45c-1186ba98f90b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be88c74ac3be4e4780e82df578aa7c43'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:14.991164 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ef23b2c1-4e66-40ec-b45c-1186ba98f90b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be88c74ac3be4e4780e82df578aa7c43'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:15.006434 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ef23b2c1-4e66-40ec-b45c-1186ba98f90b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be88c74ac3be4e4780e82df578aa7c43'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:15.014198 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1057/2121] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:56:14 2022] DELETE /identity/v3/users/be88c74ac3be4e4780e82df578aa7c43 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:15.520696 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce32c65-4fd0-4388-88f4-f4ed0763e34d admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:15.522313 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-8ce32c65-4fd0-4388-88f4-f4ed0763e34d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:15.545965 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce32c65-4fd0-4388-88f4-f4ed0763e34d admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:15.653605 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d58f0f0-55e2-45b7-b687-b7a884b2fbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:15.654554 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58f0f0-55e2-45b7-b687-b7a884b2fbc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:15.654839 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58f0f0-55e2-45b7-b687-b7a884b2fbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:15.655171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d58f0f0-55e2-45b7-b687-b7a884b2fbc2 admin admin] PATH_INFO: `/v3/projects/195f38310fb24f40be5b4a346607018d` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:15.656607 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d58f0f0-55e2-45b7-b687-b7a884b2fbc2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=195f38310fb24f40be5b4a346607018d)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:15.658553 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d58f0f0-55e2-45b7-b687-b7a884b2fbc2 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:15.835564 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-4d58f0f0-55e2-45b7-b687-b7a884b2fbc2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '195f38310fb24f40be5b4a346607018d', 'request_id': 'req-4d58f0f0-55e2-45b7-b687-b7a884b2fbc2'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:15.843746 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-4d58f0f0-55e2-45b7-b687-b7a884b2fbc2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '195f38310fb24f40be5b4a346607018d', 'request_id': 'req-4d58f0f0-55e2-45b7-b687-b7a884b2fbc2'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:15.943118 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-4d58f0f0-55e2-45b7-b687-b7a884b2fbc2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '195f38310fb24f40be5b4a346607018d', 'request_id': 'req-4d58f0f0-55e2-45b7-b687-b7a884b2fbc2'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:15.961710 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-4d58f0f0-55e2-45b7-b687-b7a884b2fbc2 admin admin] The token cache is being invalidate because project 195f38310fb24f40be5b4a346607018d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:56:15.962191 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-4d58f0f0-55e2-45b7-b687-b7a884b2fbc2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:15.963002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-4d58f0f0-55e2-45b7-b687-b7a884b2fbc2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:15.965401 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1065/2122] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:56:15 2022] DELETE /identity/v3/projects/195f38310fb24f40be5b4a346607018d => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:15.971320 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463d610-b45e-4738-91aa-ed517e468ebd None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:15.973564 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463d610-b45e-4738-91aa-ed517e468ebd None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:15.973564 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7463d610-b45e-4738-91aa-ed517e468ebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:15.998800 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-7463d610-b45e-4738-91aa-ed517e468ebd None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:16.001233 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-7463d610-b45e-4738-91aa-ed517e468ebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:16.149716 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1058/2123] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:15 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:16.152214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e837e03-479d-4b31-b68b-e14db4dfb587 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:16.163205 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e837e03-479d-4b31-b68b-e14db4dfb587 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:16.163205 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e837e03-479d-4b31-b68b-e14db4dfb587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:16.163920 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:16.179271 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-4e837e03-479d-4b31-b68b-e14db4dfb587 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:16.180559 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-4e837e03-479d-4b31-b68b-e14db4dfb587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:16.192119 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1066/2124] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:16 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:16.198220 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463d610-b45e-4738-91aa-ed517e468ebd None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:16.221851 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7463d610-b45e-4738-91aa-ed517e468ebd None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:16.227403 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd45c9-4b4f-4a16-8e0a-b2d059bec1ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:16.228384 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd45c9-4b4f-4a16-8e0a-b2d059bec1ad admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:16.228811 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd45c9-4b4f-4a16-8e0a-b2d059bec1ad admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:16.229257 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addd45c9-4b4f-4a16-8e0a-b2d059bec1ad admin admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:16.230102 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addd45c9-4b4f-4a16-8e0a-b2d059bec1ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:56:16.230563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addd45c9-4b4f-4a16-8e0a-b2d059bec1ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:16.233481 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addd45c9-4b4f-4a16-8e0a-b2d059bec1ad admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:16.248082 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1059/2125] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:56:16 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:56:16.252666 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e837e03-479d-4b31-b68b-e14db4dfb587 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:16.263822 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e837e03-479d-4b31-b68b-e14db4dfb587 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:16.269733 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af9cce-7694-477d-80fa-a80f5d9fdb56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:16.270577 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af9cce-7694-477d-80fa-a80f5d9fdb56 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:16.271031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af9cce-7694-477d-80fa-a80f5d9fdb56 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:16.271441 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90af9cce-7694-477d-80fa-a80f5d9fdb56 admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:16.272630 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90af9cce-7694-477d-80fa-a80f5d9fdb56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:16.276247 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90af9cce-7694-477d-80fa-a80f5d9fdb56 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:16.390055 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1067/2126] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:56:16 2022] POST /identity/v3/projects => generated 366 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:56:16.390518 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd45c9-4b4f-4a16-8e0a-b2d059bec1ad admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:16.406175 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addd45c9-4b4f-4a16-8e0a-b2d059bec1ad admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:16.412515 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:16.486873 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef28f6-718c-40a2-9aa6-1c2d246fd9db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:16.487819 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef28f6-718c-40a2-9aa6-1c2d246fd9db admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:16.488101 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef28f6-718c-40a2-9aa6-1c2d246fd9db admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:16.488559 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef28f6-718c-40a2-9aa6-1c2d246fd9db admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:16.489239 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef28f6-718c-40a2-9aa6-1c2d246fd9db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:16.491375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef28f6-718c-40a2-9aa6-1c2d246fd9db admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:16.626734 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1060/2127] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:56:16 2022] POST /identity/v3/users => generated 284 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:56:16.631623 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af9cce-7694-477d-80fa-a80f5d9fdb56 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:16.644345 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90af9cce-7694-477d-80fa-a80f5d9fdb56 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:16.649691 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b658c-11f0-4480-bd4f-060db4c0fa71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:16.651343 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b658c-11f0-4480-bd4f-060db4c0fa71 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:16.652037 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b658c-11f0-4480-bd4f-060db4c0fa71 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:16.652355 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b658c-11f0-4480-bd4f-060db4c0fa71 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:16.654015 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b658c-11f0-4480-bd4f-060db4c0fa71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:16.656446 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5b658c-11f0-4480-bd4f-060db4c0fa71 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:16.672511 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1068/2128] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:16 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:56:16.673411 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef28f6-718c-40a2-9aa6-1c2d246fd9db admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:16.688161 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef28f6-718c-40a2-9aa6-1c2d246fd9db admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:16.703886 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c136389-9d42-48c8-b0ce-1fe502c38d68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:16.704915 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c136389-9d42-48c8-b0ce-1fe502c38d68 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:16.705397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c136389-9d42-48c8-b0ce-1fe502c38d68 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:16.705690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c136389-9d42-48c8-b0ce-1fe502c38d68 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:16.706507 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c136389-9d42-48c8-b0ce-1fe502c38d68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:16.708577 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c136389-9d42-48c8-b0ce-1fe502c38d68 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:16.722133 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1061/2129] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:16 2022] GET /identity/v3/roles => generated 1427 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:56:16.726411 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b658c-11f0-4480-bd4f-060db4c0fa71 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:16.738930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5b658c-11f0-4480-bd4f-060db4c0fa71 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:16.744753 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c33a7-c5c4-4b31-a13f-19e63687afc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:16.745588 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c33a7-c5c4-4b31-a13f-19e63687afc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:16.746856 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c33a7-c5c4-4b31-a13f-19e63687afc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:16.747227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c33a7-c5c4-4b31-a13f-19e63687afc2 admin admin] PATH_INFO: `/v3/projects/38b74d72cf2f435aae834cd13c4d222e/users/c12e07a3f118492b875526a712ba5b40/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:16.763550 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c33a7-c5c4-4b31-a13f-19e63687afc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38b74d72cf2f435aae834cd13c4d222e, user_id=c12e07a3f118492b875526a712ba5b40, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:16.766339 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c33a7-c5c4-4b31-a13f-19e63687afc2 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:16.891683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1069/2130] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:56:16 2022] PUT /identity/v3/projects/38b74d72cf2f435aae834cd13c4d222e/users/c12e07a3f118492b875526a712ba5b40/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:16.901865 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d3075-652d-4d3e-8c30-5a741e797c88 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:16.902258 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d3075-652d-4d3e-8c30-5a741e797c88 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:16.903272 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249d3075-652d-4d3e-8c30-5a741e797c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:16.932969 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-249d3075-652d-4d3e-8c30-5a741e797c88 None None] MFA Rules not processed for user `c12e07a3f118492b875526a712ba5b40`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:16.934774 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-249d3075-652d-4d3e-8c30-5a741e797c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:17.019655 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1062/2131] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:17.024447 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c33a7-c5c4-4b31-a13f-19e63687afc2 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:17.096820 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c33a7-c5c4-4b31-a13f-19e63687afc2 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:17.102540 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769abf5a-550e-436c-8399-3bf10864b89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:17.103282 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769abf5a-550e-436c-8399-3bf10864b89a admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:17.103763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769abf5a-550e-436c-8399-3bf10864b89a admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:17.104052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769abf5a-550e-436c-8399-3bf10864b89a admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:17.104863 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769abf5a-550e-436c-8399-3bf10864b89a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:17.107913 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769abf5a-550e-436c-8399-3bf10864b89a admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:17.256402 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1070/2132] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:56:17 2022] POST /identity/v3/projects => generated 370 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:56:17.260354 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d3075-652d-4d3e-8c30-5a741e797c88 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:17.272926 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249d3075-652d-4d3e-8c30-5a741e797c88 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:17.391100 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb170630-6457-42b3-a77a-17f86a5f7c92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:17.391100 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb170630-6457-42b3-a77a-17f86a5f7c92 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:17.391100 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb170630-6457-42b3-a77a-17f86a5f7c92 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:17.391100 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb170630-6457-42b3-a77a-17f86a5f7c92 admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:17.391393 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb170630-6457-42b3-a77a-17f86a5f7c92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:17.392339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb170630-6457-42b3-a77a-17f86a5f7c92 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:17.492114 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1063/2133] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:56:17 2022] POST /identity/v3/users => generated 292 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:56:17.496758 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769abf5a-550e-436c-8399-3bf10864b89a admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:17.512857 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769abf5a-550e-436c-8399-3bf10864b89a admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:17.517957 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42313463-4a81-4529-8d7c-a7698dc208d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:17.518661 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42313463-4a81-4529-8d7c-a7698dc208d6 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:17.519021 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42313463-4a81-4529-8d7c-a7698dc208d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:17.519456 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42313463-4a81-4529-8d7c-a7698dc208d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:17.520231 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42313463-4a81-4529-8d7c-a7698dc208d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:17.521908 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42313463-4a81-4529-8d7c-a7698dc208d6 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:17.536081 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1071/2134] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:17 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:56:17.540666 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb170630-6457-42b3-a77a-17f86a5f7c92 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:17.551940 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb170630-6457-42b3-a77a-17f86a5f7c92 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:17.559952 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96822306-5d71-4c90-bb69-d96cce5060e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:17.560873 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96822306-5d71-4c90-bb69-d96cce5060e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:17.561305 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96822306-5d71-4c90-bb69-d96cce5060e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:17.561727 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96822306-5d71-4c90-bb69-d96cce5060e9 admin admin] PATH_INFO: `/v3/projects/31435604708146dea241bad3ea1a7fa1/users/7a82df228ca94779be94c3e1f037322f/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:17.585843 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96822306-5d71-4c90-bb69-d96cce5060e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31435604708146dea241bad3ea1a7fa1, user_id=7a82df228ca94779be94c3e1f037322f, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:17.589982 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96822306-5d71-4c90-bb69-d96cce5060e9 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:17.691295 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1064/2135] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:56:17 2022] PUT /identity/v3/projects/31435604708146dea241bad3ea1a7fa1/users/7a82df228ca94779be94c3e1f037322f/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:17.693885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42313463-4a81-4529-8d7c-a7698dc208d6 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:17.711625 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42313463-4a81-4529-8d7c-a7698dc208d6 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:17.799572 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f1744f-5a7c-471f-9fe3-f0e6688082e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:17.800330 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1744f-5a7c-471f-9fe3-f0e6688082e6 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:17.800672 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1744f-5a7c-471f-9fe3-f0e6688082e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:17.801006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f1744f-5a7c-471f-9fe3-f0e6688082e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:17.801834 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1744f-5a7c-471f-9fe3-f0e6688082e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:17.805774 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f1744f-5a7c-471f-9fe3-f0e6688082e6 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:17.815630 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1072/2136] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:17 2022] GET /identity/v3/roles => generated 1427 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:56:17.819869 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96822306-5d71-4c90-bb69-d96cce5060e9 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:17.832260 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96822306-5d71-4c90-bb69-d96cce5060e9 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:17.837593 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75741654-39d7-428a-89ac-e5ddd4475089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:17.838539 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75741654-39d7-428a-89ac-e5ddd4475089 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:17.838919 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75741654-39d7-428a-89ac-e5ddd4475089 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:17.839316 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75741654-39d7-428a-89ac-e5ddd4475089 admin admin] PATH_INFO: `/v3/projects/31435604708146dea241bad3ea1a7fa1/users/7a82df228ca94779be94c3e1f037322f/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:17.842673 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75741654-39d7-428a-89ac-e5ddd4475089 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31435604708146dea241bad3ea1a7fa1, user_id=7a82df228ca94779be94c3e1f037322f, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:17.845611 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75741654-39d7-428a-89ac-e5ddd4475089 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:17.873409 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1065/2137] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:56:17 2022] PUT /identity/v3/projects/31435604708146dea241bad3ea1a7fa1/users/7a82df228ca94779be94c3e1f037322f/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:17.884186 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ea3df-71d3-4e98-8153-a9e9b3b1e807 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:17.884485 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ea3df-71d3-4e98-8153-a9e9b3b1e807 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:17.884766 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ea3df-71d3-4e98-8153-a9e9b3b1e807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:17.926806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-ff6ea3df-71d3-4e98-8153-a9e9b3b1e807 None None] MFA Rules not processed for user `7a82df228ca94779be94c3e1f037322f`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:17.928709 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-ff6ea3df-71d3-4e98-8153-a9e9b3b1e807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:18.039475 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1073/2138] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:17 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:18.051847 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75741654-39d7-428a-89ac-e5ddd4475089 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:18.054141 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-75741654-39d7-428a-89ac-e5ddd4475089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:18.127899 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75741654-39d7-428a-89ac-e5ddd4475089 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:18.131879 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ed7748-7811-4775-b78d-598381728e48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:18.132493 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed7748-7811-4775-b78d-598381728e48 service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:18.132772 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed7748-7811-4775-b78d-598381728e48 service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:18.133046 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ed7748-7811-4775-b78d-598381728e48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:18.143991 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ed7748-7811-4775-b78d-598381728e48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:18.145479 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ed7748-7811-4775-b78d-598381728e48 service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:18.156697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1066/2139] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:56:18 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:18.393045 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:18.498404 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:20.748167 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fcb42-0cf1-46dc-8d63-d0d276fd06da None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:20.748738 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fcb42-0cf1-46dc-8d63-d0d276fd06da None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:20.748965 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7fcb42-0cf1-46dc-8d63-d0d276fd06da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:20.791453 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-dd7fcb42-0cf1-46dc-8d63-d0d276fd06da None None] MFA Rules not processed for user `3057a53debc34c829372c03567e1119d`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:20.793573 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-dd7fcb42-0cf1-46dc-8d63-d0d276fd06da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:20.876971 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1074/2140] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:20 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:20.889026 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b48ffc-931b-46cd-ade3-6c3a2c459833 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:20.890168 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b48ffc-931b-46cd-ade3-6c3a2c459833 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:20.890647 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b48ffc-931b-46cd-ade3-6c3a2c459833 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:20.913003 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-31b48ffc-931b-46cd-ade3-6c3a2c459833 None None] MFA Rules not processed for user `3057a53debc34c829372c03567e1119d`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:20.914380 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-31b48ffc-931b-46cd-ade3-6c3a2c459833 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:20.925100 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1067/2141] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:20 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:56:20.946024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fcb42-0cf1-46dc-8d63-d0d276fd06da None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:20.948298 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-dd7fcb42-0cf1-46dc-8d63-d0d276fd06da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:21.088902 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7fcb42-0cf1-46dc-8d63-d0d276fd06da None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:21.095837 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b641e5-3de6-4558-82f0-7b0f3c448217 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:21.096591 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b641e5-3de6-4558-82f0-7b0f3c448217 service nova] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:21.096943 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b641e5-3de6-4558-82f0-7b0f3c448217 service nova] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:21.097297 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b641e5-3de6-4558-82f0-7b0f3c448217 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:21.107251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b641e5-3de6-4558-82f0-7b0f3c448217 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:21.109265 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b641e5-3de6-4558-82f0-7b0f3c448217 service nova] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:21.120512 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1075/2142] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:56:20 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:22.015174 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b14e7-ff38-4aa4-b94a-b2e137ceab3b None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:22.015698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b14e7-ff38-4aa4-b94a-b2e137ceab3b None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:22.015791 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1b14e7-ff38-4aa4-b94a-b2e137ceab3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:22.048259 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-6a1b14e7-ff38-4aa4-b94a-b2e137ceab3b None None] MFA Rules not processed for user `3057a53debc34c829372c03567e1119d`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:22.049802 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-6a1b14e7-ff38-4aa4-b94a-b2e137ceab3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:22.060211 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1068/2143] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:22 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:22.714775 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b641e5-3de6-4558-82f0-7b0f3c448217 service nova] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:22.736473 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:22.823814 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b641e5-3de6-4558-82f0-7b0f3c448217 service nova] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:22.829924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eafa38-56a7-4c8a-ad2f-69dbc4c5cb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:22.830731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eafa38-56a7-4c8a-ad2f-69dbc4c5cb5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:22.831161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eafa38-56a7-4c8a-ad2f-69dbc4c5cb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:22.831539 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8eafa38-56a7-4c8a-ad2f-69dbc4c5cb5d admin admin] PATH_INFO: `/v3/users/c12e07a3f118492b875526a712ba5b40` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:22.833314 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eafa38-56a7-4c8a-ad2f-69dbc4c5cb5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c12e07a3f118492b875526a712ba5b40)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:22.835973 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8eafa38-56a7-4c8a-ad2f-69dbc4c5cb5d admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:23.070616 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-b8eafa38-56a7-4c8a-ad2f-69dbc4c5cb5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c12e07a3f118492b875526a712ba5b40'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:23.071925 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-b8eafa38-56a7-4c8a-ad2f-69dbc4c5cb5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c12e07a3f118492b875526a712ba5b40'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:23.188222 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-b8eafa38-56a7-4c8a-ad2f-69dbc4c5cb5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c12e07a3f118492b875526a712ba5b40'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:23.196527 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-b8eafa38-56a7-4c8a-ad2f-69dbc4c5cb5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c12e07a3f118492b875526a712ba5b40'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:23.209882 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-b8eafa38-56a7-4c8a-ad2f-69dbc4c5cb5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c12e07a3f118492b875526a712ba5b40'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:23.217804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1076/2144] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:56:22 2022] DELETE /identity/v3/users/c12e07a3f118492b875526a712ba5b40 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:23.222629 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054a32e-0c72-4dc9-8046-6c4ce7d010e4 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:23.223146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054a32e-0c72-4dc9-8046-6c4ce7d010e4 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:23.223271 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054a32e-0c72-4dc9-8046-6c4ce7d010e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:23.245691 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-6054a32e-0c72-4dc9-8046-6c4ce7d010e4 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:23.247917 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-6054a32e-0c72-4dc9-8046-6c4ce7d010e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:23.346340 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1069/2145] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:23 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:23.357178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eafa38-56a7-4c8a-ad2f-69dbc4c5cb5d admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:23.359575 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-b8eafa38-56a7-4c8a-ad2f-69dbc4c5cb5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:23.444492 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8eafa38-56a7-4c8a-ad2f-69dbc4c5cb5d admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:23.451757 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1152436-5955-4b4a-91f4-079eff828823 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:23.452605 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1152436-5955-4b4a-91f4-079eff828823 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:23.453025 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1152436-5955-4b4a-91f4-079eff828823 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:23.453459 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1152436-5955-4b4a-91f4-079eff828823 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:23.468837 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1152436-5955-4b4a-91f4-079eff828823 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:23.470727 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1152436-5955-4b4a-91f4-079eff828823 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:23.480925 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1077/2146] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:56:23 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:24.009027 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054a32e-0c72-4dc9-8046-6c4ce7d010e4 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:24.010770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-6054a32e-0c72-4dc9-8046-6c4ce7d010e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:24.027757 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6054a32e-0c72-4dc9-8046-6c4ce7d010e4 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:24.033021 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70453032-22b3-4d84-92a3-5d6c81a17040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:24.033904 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70453032-22b3-4d84-92a3-5d6c81a17040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:24.034441 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70453032-22b3-4d84-92a3-5d6c81a17040 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:24.034772 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70453032-22b3-4d84-92a3-5d6c81a17040 admin admin] PATH_INFO: `/v3/projects/38b74d72cf2f435aae834cd13c4d222e` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:24.036152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70453032-22b3-4d84-92a3-5d6c81a17040 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38b74d72cf2f435aae834cd13c4d222e)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:24.038522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70453032-22b3-4d84-92a3-5d6c81a17040 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:24.220070 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-70453032-22b3-4d84-92a3-5d6c81a17040 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38b74d72cf2f435aae834cd13c4d222e', 'request_id': 'req-70453032-22b3-4d84-92a3-5d6c81a17040'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:24.224938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:24.339510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-70453032-22b3-4d84-92a3-5d6c81a17040 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38b74d72cf2f435aae834cd13c4d222e', 'request_id': 'req-70453032-22b3-4d84-92a3-5d6c81a17040'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:24.351978 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-70453032-22b3-4d84-92a3-5d6c81a17040 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38b74d72cf2f435aae834cd13c4d222e', 'request_id': 'req-70453032-22b3-4d84-92a3-5d6c81a17040'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:24.374897 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-70453032-22b3-4d84-92a3-5d6c81a17040 admin admin] The token cache is being invalidate because project 38b74d72cf2f435aae834cd13c4d222e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:56:24.375439 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-70453032-22b3-4d84-92a3-5d6c81a17040 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:24.376266 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-70453032-22b3-4d84-92a3-5d6c81a17040 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:24.378265 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1070/2147] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:56:24 2022] DELETE /identity/v3/projects/38b74d72cf2f435aae834cd13c4d222e => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:24.382575 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1152436-5955-4b4a-91f4-079eff828823 service neutron] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:24.394266 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1152436-5955-4b4a-91f4-079eff828823 service neutron] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:24.441629 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:24.496664 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c0b58-5810-42dd-9ad9-15adc4044849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:24.497328 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c0b58-5810-42dd-9ad9-15adc4044849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:24.497604 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c0b58-5810-42dd-9ad9-15adc4044849 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:24.497876 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c0b58-5810-42dd-9ad9-15adc4044849 admin admin] PATH_INFO: `/v3/users/7a82df228ca94779be94c3e1f037322f` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:24.499074 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c0b58-5810-42dd-9ad9-15adc4044849 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a82df228ca94779be94c3e1f037322f)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:24.501518 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c0b58-5810-42dd-9ad9-15adc4044849 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:24.711018 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c53c0b58-5810-42dd-9ad9-15adc4044849 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a82df228ca94779be94c3e1f037322f'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:24.711792 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c53c0b58-5810-42dd-9ad9-15adc4044849 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a82df228ca94779be94c3e1f037322f'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:24.811722 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c53c0b58-5810-42dd-9ad9-15adc4044849 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a82df228ca94779be94c3e1f037322f'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:24.824960 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c53c0b58-5810-42dd-9ad9-15adc4044849 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a82df228ca94779be94c3e1f037322f'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:24.854171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c53c0b58-5810-42dd-9ad9-15adc4044849 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a82df228ca94779be94c3e1f037322f'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:24.862116 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1078/2148] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:56:24 2022] DELETE /identity/v3/users/7a82df228ca94779be94c3e1f037322f => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:56:25.200646 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70453032-22b3-4d84-92a3-5d6c81a17040 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:25.202781 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-70453032-22b3-4d84-92a3-5d6c81a17040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:25.226116 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70453032-22b3-4d84-92a3-5d6c81a17040 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:25.307247 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084701d5-4665-498e-82b2-3eb59f0a8ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:25.308298 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084701d5-4665-498e-82b2-3eb59f0a8ae6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:25.309358 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084701d5-4665-498e-82b2-3eb59f0a8ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:25.309785 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084701d5-4665-498e-82b2-3eb59f0a8ae6 admin admin] PATH_INFO: `/v3/users/4d1510bdacfb438a8b1118f9a0776b29` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:25.311691 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084701d5-4665-498e-82b2-3eb59f0a8ae6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d1510bdacfb438a8b1118f9a0776b29)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:25.314185 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084701d5-4665-498e-82b2-3eb59f0a8ae6 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:25.530440 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-084701d5-4665-498e-82b2-3eb59f0a8ae6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d1510bdacfb438a8b1118f9a0776b29'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:25.540904 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-084701d5-4665-498e-82b2-3eb59f0a8ae6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d1510bdacfb438a8b1118f9a0776b29'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:25.542094 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-084701d5-4665-498e-82b2-3eb59f0a8ae6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d1510bdacfb438a8b1118f9a0776b29'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:25.653351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-084701d5-4665-498e-82b2-3eb59f0a8ae6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d1510bdacfb438a8b1118f9a0776b29'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:25.669725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-084701d5-4665-498e-82b2-3eb59f0a8ae6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d1510bdacfb438a8b1118f9a0776b29'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:25.676397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1071/2149] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:56:25 2022] DELETE /identity/v3/users/4d1510bdacfb438a8b1118f9a0776b29 => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:25.681102 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeed535-5cb9-44f4-a155-349bef32f743 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:25.681565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeed535-5cb9-44f4-a155-349bef32f743 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:25.682741 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceeed535-5cb9-44f4-a155-349bef32f743 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:25.702352 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-ceeed535-5cb9-44f4-a155-349bef32f743 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:25.703623 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-ceeed535-5cb9-44f4-a155-349bef32f743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:25.749103 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084701d5-4665-498e-82b2-3eb59f0a8ae6 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:25.750446 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-084701d5-4665-498e-82b2-3eb59f0a8ae6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:25.780356 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1079/2150] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:56:25.794010 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeed535-5cb9-44f4-a155-349bef32f743 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:25.795867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-ceeed535-5cb9-44f4-a155-349bef32f743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:25.887788 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084701d5-4665-498e-82b2-3eb59f0a8ae6 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:25.894851 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e191d951-c414-44c0-8502-f462d7ce72e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:25.895592 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191d951-c414-44c0-8502-f462d7ce72e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:25.895869 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191d951-c414-44c0-8502-f462d7ce72e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:25.896151 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e191d951-c414-44c0-8502-f462d7ce72e4 admin admin] PATH_INFO: `/v3/projects/31435604708146dea241bad3ea1a7fa1` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:25.898150 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e191d951-c414-44c0-8502-f462d7ce72e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31435604708146dea241bad3ea1a7fa1)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:25.901046 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e191d951-c414-44c0-8502-f462d7ce72e4 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:25.904929 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceeed535-5cb9-44f4-a155-349bef32f743 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:25.910028 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950e5701-1953-4cdb-b362-073a0cb3e412 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:25.910826 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e5701-1953-4cdb-b362-073a0cb3e412 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:25.911175 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e5701-1953-4cdb-b362-073a0cb3e412 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:25.911739 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950e5701-1953-4cdb-b362-073a0cb3e412 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:25.926100 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e5701-1953-4cdb-b362-073a0cb3e412 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:25.927815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950e5701-1953-4cdb-b362-073a0cb3e412 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:25.940416 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1080/2151] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:56:25 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:26.033402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-e191d951-c414-44c0-8502-f462d7ce72e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31435604708146dea241bad3ea1a7fa1', 'request_id': 'req-e191d951-c414-44c0-8502-f462d7ce72e4'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:26.182948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-e191d951-c414-44c0-8502-f462d7ce72e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31435604708146dea241bad3ea1a7fa1', 'request_id': 'req-e191d951-c414-44c0-8502-f462d7ce72e4'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:26.193194 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-e191d951-c414-44c0-8502-f462d7ce72e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31435604708146dea241bad3ea1a7fa1', 'request_id': 'req-e191d951-c414-44c0-8502-f462d7ce72e4'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:26.206563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-e191d951-c414-44c0-8502-f462d7ce72e4 admin admin] The token cache is being invalidate because project 31435604708146dea241bad3ea1a7fa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:56:26.206852 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-e191d951-c414-44c0-8502-f462d7ce72e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:26.207554 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-e191d951-c414-44c0-8502-f462d7ce72e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:26.209143 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1072/2152] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:56:25 2022] DELETE /identity/v3/projects/31435604708146dea241bad3ea1a7fa1 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:56:26.229976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055c88a-fce8-429f-a973-9a1bc5b4982b None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:26.230585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055c88a-fce8-429f-a973-9a1bc5b4982b None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:26.232036 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c055c88a-fce8-429f-a973-9a1bc5b4982b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:26.256623 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-c055c88a-fce8-429f-a973-9a1bc5b4982b None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:26.260576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-c055c88a-fce8-429f-a973-9a1bc5b4982b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:26.360223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401b3dc-4da2-4e2b-a231-355447870085 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:26.360815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1081/2153] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:26 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:26.361246 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401b3dc-4da2-4e2b-a231-355447870085 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:26.361682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a401b3dc-4da2-4e2b-a231-355447870085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:26.418064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-a401b3dc-4da2-4e2b-a231-355447870085 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:26.420048 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-a401b3dc-4da2-4e2b-a231-355447870085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:26.448754 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1073/2154] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:26 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:26.456487 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055c88a-fce8-429f-a973-9a1bc5b4982b None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:26.468275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c055c88a-fce8-429f-a973-9a1bc5b4982b None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:26.474389 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8c668-5fed-4fdf-b5c3-d02e706c1022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:26.475414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8c668-5fed-4fdf-b5c3-d02e706c1022 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:26.476799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8c668-5fed-4fdf-b5c3-d02e706c1022 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:26.477096 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d8c668-5fed-4fdf-b5c3-d02e706c1022 admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:26.477822 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8c668-5fed-4fdf-b5c3-d02e706c1022 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:56:26.478147 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8c668-5fed-4fdf-b5c3-d02e706c1022 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:26.479787 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d8c668-5fed-4fdf-b5c3-d02e706c1022 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:26.500048 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1082/2155] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:56:26 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:56:26.505299 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401b3dc-4da2-4e2b-a231-355447870085 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:26.516690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a401b3dc-4da2-4e2b-a231-355447870085 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:26.523548 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec6cf1-c968-43c8-af82-83953cee4c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:26.524416 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec6cf1-c968-43c8-af82-83953cee4c14 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:26.524915 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec6cf1-c968-43c8-af82-83953cee4c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:26.525394 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec6cf1-c968-43c8-af82-83953cee4c14 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:26.526228 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ec6cf1-c968-43c8-af82-83953cee4c14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:26.530338 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ec6cf1-c968-43c8-af82-83953cee4c14 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:26.640585 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1074/2156] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:56:26 2022] POST /identity/v3/projects => generated 374 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:56:26.644837 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8c668-5fed-4fdf-b5c3-d02e706c1022 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:26.655662 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d8c668-5fed-4fdf-b5c3-d02e706c1022 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:26.718360 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106895e-40ee-4dfe-a13e-39d852686718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:26.719032 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106895e-40ee-4dfe-a13e-39d852686718 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:26.719297 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106895e-40ee-4dfe-a13e-39d852686718 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:26.719570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5106895e-40ee-4dfe-a13e-39d852686718 admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:26.720260 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106895e-40ee-4dfe-a13e-39d852686718 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:26.722225 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5106895e-40ee-4dfe-a13e-39d852686718 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:26.883662 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1083/2157] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:56:26 2022] POST /identity/v3/users => generated 288 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:56:26.884976 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec6cf1-c968-43c8-af82-83953cee4c14 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:26.897542 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec6cf1-c968-43c8-af82-83953cee4c14 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:26.909506 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:26.912619 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546e92c1-904a-4265-a923-ee7a7b5a31c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:26.914361 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546e92c1-904a-4265-a923-ee7a7b5a31c9 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:26.914361 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546e92c1-904a-4265-a923-ee7a7b5a31c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:26.914591 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546e92c1-904a-4265-a923-ee7a7b5a31c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:26.915349 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546e92c1-904a-4265-a923-ee7a7b5a31c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:26.919473 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546e92c1-904a-4265-a923-ee7a7b5a31c9 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:26.938437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1075/2158] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:26 2022] GET /identity/v3/roles => generated 1427 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:56:26.940076 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106895e-40ee-4dfe-a13e-39d852686718 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:26.957817 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5106895e-40ee-4dfe-a13e-39d852686718 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:26.976976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b361e5f-b527-4923-97ad-e3d893895122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:26.977629 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b361e5f-b527-4923-97ad-e3d893895122 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:26.977911 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b361e5f-b527-4923-97ad-e3d893895122 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:26.978546 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b361e5f-b527-4923-97ad-e3d893895122 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:26.979177 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b361e5f-b527-4923-97ad-e3d893895122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:26.983081 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b361e5f-b527-4923-97ad-e3d893895122 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:26.991326 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:27.020229 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1084/2159] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:26 2022] GET /identity/v3/roles => generated 1427 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:56:27.024685 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546e92c1-904a-4265-a923-ee7a7b5a31c9 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:27.039602 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546e92c1-904a-4265-a923-ee7a7b5a31c9 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:27.046414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a136908-9afa-4692-8180-90d7c42d3746 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:27.047357 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a136908-9afa-4692-8180-90d7c42d3746 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:27.047893 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a136908-9afa-4692-8180-90d7c42d3746 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:27.048335 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a136908-9afa-4692-8180-90d7c42d3746 admin admin] PATH_INFO: `/v3/projects/bcc5c2dead834573b20f117087cae1cb/users/a2ddaf1855154ddd9c2aba338ca37885/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:27.078965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a136908-9afa-4692-8180-90d7c42d3746 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcc5c2dead834573b20f117087cae1cb, user_id=a2ddaf1855154ddd9c2aba338ca37885, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:27.081942 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a136908-9afa-4692-8180-90d7c42d3746 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:27.185920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1076/2160] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:56:27 2022] PUT /identity/v3/projects/bcc5c2dead834573b20f117087cae1cb/users/a2ddaf1855154ddd9c2aba338ca37885/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:27.196323 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd0264-ba9d-40cb-a38f-8e18fc25dfdb None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:27.196748 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd0264-ba9d-40cb-a38f-8e18fc25dfdb None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:27.197492 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbd0264-ba9d-40cb-a38f-8e18fc25dfdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:27.241244 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-abbd0264-ba9d-40cb-a38f-8e18fc25dfdb None None] MFA Rules not processed for user `a2ddaf1855154ddd9c2aba338ca37885`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:27.242589 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-abbd0264-ba9d-40cb-a38f-8e18fc25dfdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:27.352154 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1085/2161] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:27.364246 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a136908-9afa-4692-8180-90d7c42d3746 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:27.366022 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-9a136908-9afa-4692-8180-90d7c42d3746 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:27.448585 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a136908-9afa-4692-8180-90d7c42d3746 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:27.456585 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34eeb1-7351-4968-ad53-a6c866423aa6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:27.457574 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34eeb1-7351-4968-ad53-a6c866423aa6 service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:27.457999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34eeb1-7351-4968-ad53-a6c866423aa6 service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:27.458432 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b34eeb1-7351-4968-ad53-a6c866423aa6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:27.468108 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34eeb1-7351-4968-ad53-a6c866423aa6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:27.477551 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b34eeb1-7351-4968-ad53-a6c866423aa6 service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:27.478575 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd0264-ba9d-40cb-a38f-8e18fc25dfdb None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:27.480143 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-abbd0264-ba9d-40cb-a38f-8e18fc25dfdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:27.480470 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1077/2162] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:56:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:27.512348 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34eeb1-7351-4968-ad53-a6c866423aa6 service cinder] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:27.557256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbd0264-ba9d-40cb-a38f-8e18fc25dfdb None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:27.564416 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c3d7d-b211-4f27-be9e-327e2b164b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:27.565066 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c3d7d-b211-4f27-be9e-327e2b164b55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:27.565344 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c3d7d-b211-4f27-be9e-327e2b164b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:27.565621 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4c3d7d-b211-4f27-be9e-327e2b164b55 admin admin] PATH_INFO: `/v3/projects/9a19b82d3ae74f0c9bd89c9cbf6533d6` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:27.566919 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4c3d7d-b211-4f27-be9e-327e2b164b55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a19b82d3ae74f0c9bd89c9cbf6533d6)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:27.568924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4c3d7d-b211-4f27-be9e-327e2b164b55 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:27.576912 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b34eeb1-7351-4968-ad53-a6c866423aa6 service cinder] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:27.581839 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d9a1f-2cfa-471a-8116-3ae7698039fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:27.582507 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d9a1f-2cfa-471a-8116-3ae7698039fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:27.582900 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d9a1f-2cfa-471a-8116-3ae7698039fc admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:27.583222 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d9a1f-2cfa-471a-8116-3ae7698039fc admin admin] PATH_INFO: `/v3/users/a2ddaf1855154ddd9c2aba338ca37885` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:27.584412 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d9a1f-2cfa-471a-8116-3ae7698039fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2ddaf1855154ddd9c2aba338ca37885)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:27.586428 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d9a1f-2cfa-471a-8116-3ae7698039fc admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:27.749646 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-ea4c3d7d-b211-4f27-be9e-327e2b164b55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a19b82d3ae74f0c9bd89c9cbf6533d6', 'request_id': 'req-ea4c3d7d-b211-4f27-be9e-327e2b164b55'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:27.756056 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-ea4c3d7d-b211-4f27-be9e-327e2b164b55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a19b82d3ae74f0c9bd89c9cbf6533d6', 'request_id': 'req-ea4c3d7d-b211-4f27-be9e-327e2b164b55'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:27.908593 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-ea4c3d7d-b211-4f27-be9e-327e2b164b55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a19b82d3ae74f0c9bd89c9cbf6533d6', 'request_id': 'req-ea4c3d7d-b211-4f27-be9e-327e2b164b55'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:27.926996 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-ea4c3d7d-b211-4f27-be9e-327e2b164b55 admin admin] The token cache is being invalidate because project 9a19b82d3ae74f0c9bd89c9cbf6533d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:56:27.927647 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-ea4c3d7d-b211-4f27-be9e-327e2b164b55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:27.928834 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-ea4c3d7d-b211-4f27-be9e-327e2b164b55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:27.930914 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-802d9a1f-2cfa-471a-8116-3ae7698039fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2ddaf1855154ddd9c2aba338ca37885'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:27.931755 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1086/2163] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:56:27 2022] DELETE /identity/v3/projects/9a19b82d3ae74f0c9bd89c9cbf6533d6 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:27.934891 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c3d7d-b211-4f27-be9e-327e2b164b55 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:27.937245 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-ea4c3d7d-b211-4f27-be9e-327e2b164b55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:27.943175 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-802d9a1f-2cfa-471a-8116-3ae7698039fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2ddaf1855154ddd9c2aba338ca37885'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:27.944169 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-802d9a1f-2cfa-471a-8116-3ae7698039fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2ddaf1855154ddd9c2aba338ca37885'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:28.008448 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4c3d7d-b211-4f27-be9e-327e2b164b55 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:28.013167 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b27b9-f8bf-4fb3-9194-f6debdf328f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:28.013804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b27b9-f8bf-4fb3-9194-f6debdf328f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:28.014086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b27b9-f8bf-4fb3-9194-f6debdf328f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:28.014363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6b27b9-f8bf-4fb3-9194-f6debdf328f2 admin admin] PATH_INFO: `/v3/users/bec9ea3f61b7483da3a629b290000fec` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:28.015732 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b27b9-f8bf-4fb3-9194-f6debdf328f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bec9ea3f61b7483da3a629b290000fec)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:28.017194 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-802d9a1f-2cfa-471a-8116-3ae7698039fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2ddaf1855154ddd9c2aba338ca37885'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:28.017624 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6b27b9-f8bf-4fb3-9194-f6debdf328f2 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:28.032597 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-802d9a1f-2cfa-471a-8116-3ae7698039fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2ddaf1855154ddd9c2aba338ca37885'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:28.040011 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1078/2164] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:56:27 2022] DELETE /identity/v3/users/a2ddaf1855154ddd9c2aba338ca37885 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:28.041786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ace7d-96c5-4fd3-b269-45e88c7f4ffd None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:28.041786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ace7d-96c5-4fd3-b269-45e88c7f4ffd None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:28.042120 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ace7d-96c5-4fd3-b269-45e88c7f4ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:28.065541 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-194ace7d-96c5-4fd3-b269-45e88c7f4ffd None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:28.066847 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-194ace7d-96c5-4fd3-b269-45e88c7f4ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:28.151683 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1079/2165] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:28.158252 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1308fb1-da04-4cc1-b3c7-af67141db419 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:28.158252 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1308fb1-da04-4cc1-b3c7-af67141db419 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:28.158252 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1308fb1-da04-4cc1-b3c7-af67141db419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:28.179254 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-e1308fb1-da04-4cc1-b3c7-af67141db419 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:28.182450 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-e1308fb1-da04-4cc1-b3c7-af67141db419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:28.194735 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1080/2166] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:28.198137 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90cec0b-4d2f-4d45-a9c7-cf56452ea759 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:28.198958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90cec0b-4d2f-4d45-a9c7-cf56452ea759 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:28.200613 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90cec0b-4d2f-4d45-a9c7-cf56452ea759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:28.212362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-3d6b27b9-f8bf-4fb3-9194-f6debdf328f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bec9ea3f61b7483da3a629b290000fec'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:28.213211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-3d6b27b9-f8bf-4fb3-9194-f6debdf328f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bec9ea3f61b7483da3a629b290000fec'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:28.221766 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-a90cec0b-4d2f-4d45-a9c7-cf56452ea759 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:28.222954 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-a90cec0b-4d2f-4d45-a9c7-cf56452ea759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:28.234207 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1081/2167] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:28.235400 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90cec0b-4d2f-4d45-a9c7-cf56452ea759 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:28.237650 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-a90cec0b-4d2f-4d45-a9c7-cf56452ea759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:28.279526 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-3d6b27b9-f8bf-4fb3-9194-f6debdf328f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bec9ea3f61b7483da3a629b290000fec'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:28.291577 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-3d6b27b9-f8bf-4fb3-9194-f6debdf328f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bec9ea3f61b7483da3a629b290000fec'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:28.309271 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-3d6b27b9-f8bf-4fb3-9194-f6debdf328f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bec9ea3f61b7483da3a629b290000fec'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:28.316722 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1087/2168] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:56:27 2022] DELETE /identity/v3/users/bec9ea3f61b7483da3a629b290000fec => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:28.320630 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b27b9-f8bf-4fb3-9194-f6debdf328f2 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:28.322027 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-3d6b27b9-f8bf-4fb3-9194-f6debdf328f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:28.331893 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90cec0b-4d2f-4d45-a9c7-cf56452ea759 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:28.404423 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff022d-d044-46a4-a4b3-85d83e43ccc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:28.405216 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff022d-d044-46a4-a4b3-85d83e43ccc4 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:28.405721 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff022d-d044-46a4-a4b3-85d83e43ccc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:28.406003 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dff022d-d044-46a4-a4b3-85d83e43ccc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:28.410766 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-8dff022d-d044-46a4-a4b3-85d83e43ccc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:28.422701 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6b27b9-f8bf-4fb3-9194-f6debdf328f2 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:28.428757 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dff022d-d044-46a4-a4b3-85d83e43ccc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:28.430457 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dff022d-d044-46a4-a4b3-85d83e43ccc4 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:28.436681 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93214fec-52a5-4638-bc56-1a0800314d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:28.437566 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93214fec-52a5-4638-bc56-1a0800314d89 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:28.437918 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93214fec-52a5-4638-bc56-1a0800314d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:28.438708 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93214fec-52a5-4638-bc56-1a0800314d89 admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:28.439087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93214fec-52a5-4638-bc56-1a0800314d89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:56:28.439650 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93214fec-52a5-4638-bc56-1a0800314d89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:28.448140 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1082/2169] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:56:28 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:28.448807 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93214fec-52a5-4638-bc56-1a0800314d89 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:28.477440 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1088/2170] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:56:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:56:28.482221 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff022d-d044-46a4-a4b3-85d83e43ccc4 service neutron] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:28.494062 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dff022d-d044-46a4-a4b3-85d83e43ccc4 service neutron] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:28.500332 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a7750b-bae5-47cd-999e-74b0cb07d1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:28.501604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7750b-bae5-47cd-999e-74b0cb07d1ef admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:28.501894 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7750b-bae5-47cd-999e-74b0cb07d1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:28.502169 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a7750b-bae5-47cd-999e-74b0cb07d1ef admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:28.502853 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a7750b-bae5-47cd-999e-74b0cb07d1ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:28.504815 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a7750b-bae5-47cd-999e-74b0cb07d1ef admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:28.591377 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:28.600354 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1083/2171] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:56:28 2022] POST /identity/v3/projects => generated 390 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:56:28.605631 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93214fec-52a5-4638-bc56-1a0800314d89 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:28.623337 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93214fec-52a5-4638-bc56-1a0800314d89 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:28.704189 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d4135-fae8-4a87-a1cf-9b8aca91ab06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:28.704924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d4135-fae8-4a87-a1cf-9b8aca91ab06 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:28.705396 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d4135-fae8-4a87-a1cf-9b8aca91ab06 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:28.705819 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23d4135-fae8-4a87-a1cf-9b8aca91ab06 admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:28.706732 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d4135-fae8-4a87-a1cf-9b8aca91ab06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:28.708739 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23d4135-fae8-4a87-a1cf-9b8aca91ab06 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:28.743199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:28.829516 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1089/2172] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:56:28 2022] POST /identity/v3/users => generated 296 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:56:28.834292 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a7750b-bae5-47cd-999e-74b0cb07d1ef admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:28.847696 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a7750b-bae5-47cd-999e-74b0cb07d1ef admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:28.852538 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac017fd1-8490-4132-80a5-1e66c3dca1a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:28.853158 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac017fd1-8490-4132-80a5-1e66c3dca1a2 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:28.856167 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac017fd1-8490-4132-80a5-1e66c3dca1a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:28.856167 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac017fd1-8490-4132-80a5-1e66c3dca1a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:28.856167 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac017fd1-8490-4132-80a5-1e66c3dca1a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:28.856167 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac017fd1-8490-4132-80a5-1e66c3dca1a2 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:28.863008 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d4135-fae8-4a87-a1cf-9b8aca91ab06 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:28.864765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-d23d4135-fae8-4a87-a1cf-9b8aca91ab06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:28.878693 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23d4135-fae8-4a87-a1cf-9b8aca91ab06 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:28.886198 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1084/2173] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:28 2022] GET /identity/v3/roles => generated 1427 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:56:28.888099 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d79ca-9b1c-4d94-a161-a05c33f354a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:28.888580 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac017fd1-8490-4132-80a5-1e66c3dca1a2 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:28.888891 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d79ca-9b1c-4d94-a161-a05c33f354a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:28.889218 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d79ca-9b1c-4d94-a161-a05c33f354a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:28.889512 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d79ca-9b1c-4d94-a161-a05c33f354a6 admin admin] PATH_INFO: `/v3/projects/b7965b49eb5547cc8de3f28b8c53f341` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:28.890751 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d79ca-9b1c-4d94-a161-a05c33f354a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7965b49eb5547cc8de3f28b8c53f341)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:28.892902 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0d79ca-9b1c-4d94-a161-a05c33f354a6 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:28.930773 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac017fd1-8490-4132-80a5-1e66c3dca1a2 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:28.938495 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33188de6-20f3-4eb6-8e26-fbda22911c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:28.939392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33188de6-20f3-4eb6-8e26-fbda22911c8e admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:28.939592 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33188de6-20f3-4eb6-8e26-fbda22911c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:28.939880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33188de6-20f3-4eb6-8e26-fbda22911c8e admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:28.940631 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33188de6-20f3-4eb6-8e26-fbda22911c8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:28.942310 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33188de6-20f3-4eb6-8e26-fbda22911c8e admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:28.953731 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1085/2174] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:28 2022] GET /identity/v3/roles => generated 1427 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:56:28.957870 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33188de6-20f3-4eb6-8e26-fbda22911c8e admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:28.968271 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33188de6-20f3-4eb6-8e26-fbda22911c8e admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:28.973805 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1d8622-9d48-4a56-a71d-2d34e9720bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:28.974689 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d8622-9d48-4a56-a71d-2d34e9720bb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:28.979557 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d8622-9d48-4a56-a71d-2d34e9720bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:28.979557 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1d8622-9d48-4a56-a71d-2d34e9720bb2 admin admin] PATH_INFO: `/v3/projects/6d6b474d83d54803b15bdfd895f0b960/users/e0618aad2b69470c9f8607d2f5925eea/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:28.992245 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d8622-9d48-4a56-a71d-2d34e9720bb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d6b474d83d54803b15bdfd895f0b960, user_id=e0618aad2b69470c9f8607d2f5925eea, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:28.995455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1d8622-9d48-4a56-a71d-2d34e9720bb2 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:29.102374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1086/2175] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:56:28 2022] PUT /identity/v3/projects/6d6b474d83d54803b15bdfd895f0b960/users/e0618aad2b69470c9f8607d2f5925eea/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:56:29.119763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-be0d79ca-9b1c-4d94-a161-a05c33f354a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7965b49eb5547cc8de3f28b8c53f341', 'request_id': 'req-be0d79ca-9b1c-4d94-a161-a05c33f354a6'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:29.120153 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fbe79-925e-44ac-8809-6c31307d5b8f None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:29.120605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fbe79-925e-44ac-8809-6c31307d5b8f None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:29.122622 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726fbe79-925e-44ac-8809-6c31307d5b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:29.128081 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-be0d79ca-9b1c-4d94-a161-a05c33f354a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7965b49eb5547cc8de3f28b8c53f341', 'request_id': 'req-be0d79ca-9b1c-4d94-a161-a05c33f354a6'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:29.160758 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-726fbe79-925e-44ac-8809-6c31307d5b8f None None] MFA Rules not processed for user `e0618aad2b69470c9f8607d2f5925eea`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:29.162248 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-726fbe79-925e-44ac-8809-6c31307d5b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:29.223627 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-be0d79ca-9b1c-4d94-a161-a05c33f354a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7965b49eb5547cc8de3f28b8c53f341', 'request_id': 'req-be0d79ca-9b1c-4d94-a161-a05c33f354a6'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:29.241377 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-be0d79ca-9b1c-4d94-a161-a05c33f354a6 admin admin] The token cache is being invalidate because project b7965b49eb5547cc8de3f28b8c53f341 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:56:29.241683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-be0d79ca-9b1c-4d94-a161-a05c33f354a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:29.243436 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-be0d79ca-9b1c-4d94-a161-a05c33f354a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:29.246573 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1090/2176] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:56:28 2022] DELETE /identity/v3/projects/b7965b49eb5547cc8de3f28b8c53f341 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:29.251837 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d79ca-9b1c-4d94-a161-a05c33f354a6 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:29.256057 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-be0d79ca-9b1c-4d94-a161-a05c33f354a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:29.302016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1087/2177] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:29 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:29.318674 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726fbe79-925e-44ac-8809-6c31307d5b8f None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:29.320837 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-726fbe79-925e-44ac-8809-6c31307d5b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:29.356595 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d79ca-9b1c-4d94-a161-a05c33f354a6 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:29.362877 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e93b15-817f-4805-8209-ccf437df2402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:29.363574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e93b15-817f-4805-8209-ccf437df2402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:29.363861 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e93b15-817f-4805-8209-ccf437df2402 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:29.364143 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e93b15-817f-4805-8209-ccf437df2402 admin admin] PATH_INFO: `/v3/users/7be4e9c4be7b49d08923f0f9c4b7dca0` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:29.365362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e93b15-817f-4805-8209-ccf437df2402 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7be4e9c4be7b49d08923f0f9c4b7dca0)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:29.367441 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e93b15-817f-4805-8209-ccf437df2402 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:29.410176 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726fbe79-925e-44ac-8809-6c31307d5b8f None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:29.420693 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad61c191-743e-4613-96fb-18dd673affb5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:29.421351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61c191-743e-4613-96fb-18dd673affb5 service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:29.421637 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61c191-743e-4613-96fb-18dd673affb5 service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:29.421917 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad61c191-743e-4613-96fb-18dd673affb5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:29.437761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad61c191-743e-4613-96fb-18dd673affb5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:29.439534 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad61c191-743e-4613-96fb-18dd673affb5 service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:29.453539 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1088/2178] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:56:29 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:56:29.454417 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad61c191-743e-4613-96fb-18dd673affb5 service cinder] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:29.456035 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-ad61c191-743e-4613-96fb-18dd673affb5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:29.471541 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad61c191-743e-4613-96fb-18dd673affb5 service cinder] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:29.476477 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238320c5-af21-4fd3-9c0e-3e3f1588fd7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:29.477150 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238320c5-af21-4fd3-9c0e-3e3f1588fd7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:29.477438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238320c5-af21-4fd3-9c0e-3e3f1588fd7f admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:29.477875 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238320c5-af21-4fd3-9c0e-3e3f1588fd7f admin admin] PATH_INFO: `/v3/projects/bcc5c2dead834573b20f117087cae1cb` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:29.479099 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238320c5-af21-4fd3-9c0e-3e3f1588fd7f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcc5c2dead834573b20f117087cae1cb)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:29.481099 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238320c5-af21-4fd3-9c0e-3e3f1588fd7f admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:29.548649 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e7e93b15-817f-4805-8209-ccf437df2402 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7be4e9c4be7b49d08923f0f9c4b7dca0'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:29.550125 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e7e93b15-817f-4805-8209-ccf437df2402 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7be4e9c4be7b49d08923f0f9c4b7dca0'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:29.605790 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-238320c5-af21-4fd3-9c0e-3e3f1588fd7f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcc5c2dead834573b20f117087cae1cb', 'request_id': 'req-238320c5-af21-4fd3-9c0e-3e3f1588fd7f'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:29.628843 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e7e93b15-817f-4805-8209-ccf437df2402 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7be4e9c4be7b49d08923f0f9c4b7dca0'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:29.636157 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e7e93b15-817f-4805-8209-ccf437df2402 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7be4e9c4be7b49d08923f0f9c4b7dca0'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:29.648722 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e7e93b15-817f-4805-8209-ccf437df2402 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7be4e9c4be7b49d08923f0f9c4b7dca0'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:29.654770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1091/2179] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:56:29 2022] DELETE /identity/v3/users/7be4e9c4be7b49d08923f0f9c4b7dca0 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:29.759661 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-238320c5-af21-4fd3-9c0e-3e3f1588fd7f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcc5c2dead834573b20f117087cae1cb', 'request_id': 'req-238320c5-af21-4fd3-9c0e-3e3f1588fd7f'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:29.771996 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-238320c5-af21-4fd3-9c0e-3e3f1588fd7f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcc5c2dead834573b20f117087cae1cb', 'request_id': 'req-238320c5-af21-4fd3-9c0e-3e3f1588fd7f'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:29.788098 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-238320c5-af21-4fd3-9c0e-3e3f1588fd7f admin admin] The token cache is being invalidate because project bcc5c2dead834573b20f117087cae1cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:56:29.788395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-238320c5-af21-4fd3-9c0e-3e3f1588fd7f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:29.789089 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-238320c5-af21-4fd3-9c0e-3e3f1588fd7f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:29.791024 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1089/2180] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:56:29 2022] DELETE /identity/v3/projects/bcc5c2dead834573b20f117087cae1cb => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:29.823711 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:29.965078 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e93b15-817f-4805-8209-ccf437df2402 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:29.967685 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-e7e93b15-817f-4805-8209-ccf437df2402 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:30.070469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e93b15-817f-4805-8209-ccf437df2402 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:30.080053 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37245d0d-0505-45aa-ba5b-9fdbae3d7d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:30.080929 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37245d0d-0505-45aa-ba5b-9fdbae3d7d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:30.081223 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37245d0d-0505-45aa-ba5b-9fdbae3d7d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:30.081502 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37245d0d-0505-45aa-ba5b-9fdbae3d7d67 admin admin] PATH_INFO: `/v3/projects/1c276ee2728944f3987c76ccbe02a0d0` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:30.082721 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37245d0d-0505-45aa-ba5b-9fdbae3d7d67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c276ee2728944f3987c76ccbe02a0d0)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:30.085431 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37245d0d-0505-45aa-ba5b-9fdbae3d7d67 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:30.267834 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-37245d0d-0505-45aa-ba5b-9fdbae3d7d67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c276ee2728944f3987c76ccbe02a0d0', 'request_id': 'req-37245d0d-0505-45aa-ba5b-9fdbae3d7d67'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:30.275132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-37245d0d-0505-45aa-ba5b-9fdbae3d7d67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c276ee2728944f3987c76ccbe02a0d0', 'request_id': 'req-37245d0d-0505-45aa-ba5b-9fdbae3d7d67'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:30.321625 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:30.392769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-37245d0d-0505-45aa-ba5b-9fdbae3d7d67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c276ee2728944f3987c76ccbe02a0d0', 'request_id': 'req-37245d0d-0505-45aa-ba5b-9fdbae3d7d67'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:30.404377 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-37245d0d-0505-45aa-ba5b-9fdbae3d7d67 admin admin] The token cache is being invalidate because project 1c276ee2728944f3987c76ccbe02a0d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:56:30.404721 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-37245d0d-0505-45aa-ba5b-9fdbae3d7d67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:30.405301 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-37245d0d-0505-45aa-ba5b-9fdbae3d7d67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:30.406837 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1092/2181] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:56:29 2022] DELETE /identity/v3/projects/1c276ee2728944f3987c76ccbe02a0d0 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:30.411959 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a4f53-79f1-4bd7-83ac-3b2ef4b98850 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:30.412543 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a4f53-79f1-4bd7-83ac-3b2ef4b98850 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:30.412543 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06a4f53-79f1-4bd7-83ac-3b2ef4b98850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:30.435182 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-f06a4f53-79f1-4bd7-83ac-3b2ef4b98850 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:30.436504 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f06a4f53-79f1-4bd7-83ac-3b2ef4b98850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:30.498401 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1090/2182] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:30.506516 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf92d5-d993-43ed-ae2f-06bd8adec5ae None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:30.506822 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf92d5-d993-43ed-ae2f-06bd8adec5ae None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:30.507130 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bf92d5-d993-43ed-ae2f-06bd8adec5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:30.527540 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-30bf92d5-d993-43ed-ae2f-06bd8adec5ae None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:30.528783 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-30bf92d5-d993-43ed-ae2f-06bd8adec5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:30.537313 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1093/2183] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:30 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:30.541597 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a4f53-79f1-4bd7-83ac-3b2ef4b98850 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:30.552264 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06a4f53-79f1-4bd7-83ac-3b2ef4b98850 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:30.556934 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82d7bc-dc72-4b1b-a5fa-ce566fb878ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:30.557542 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82d7bc-dc72-4b1b-a5fa-ce566fb878ad admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:30.557842 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82d7bc-dc72-4b1b-a5fa-ce566fb878ad admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:30.558125 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82d7bc-dc72-4b1b-a5fa-ce566fb878ad admin admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:30.564493 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b82d7bc-dc72-4b1b-a5fa-ce566fb878ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:56:30.564493 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b82d7bc-dc72-4b1b-a5fa-ce566fb878ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:30.564493 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b82d7bc-dc72-4b1b-a5fa-ce566fb878ad admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:30.572351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1091/2184] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:56:30 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:56:30.576854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf92d5-d993-43ed-ae2f-06bd8adec5ae None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:30.586720 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bf92d5-d993-43ed-ae2f-06bd8adec5ae None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:30.592325 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25421ebc-a38f-4af0-8764-f06b58a39267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:30.592941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25421ebc-a38f-4af0-8764-f06b58a39267 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:30.593251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25421ebc-a38f-4af0-8764-f06b58a39267 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:30.593524 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25421ebc-a38f-4af0-8764-f06b58a39267 admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:30.598946 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25421ebc-a38f-4af0-8764-f06b58a39267 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:30.598946 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25421ebc-a38f-4af0-8764-f06b58a39267 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:30.738521 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1094/2185] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:56:30 2022] POST /identity/v3/projects => generated 384 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:56:30.742742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82d7bc-dc72-4b1b-a5fa-ce566fb878ad admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:30.752909 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82d7bc-dc72-4b1b-a5fa-ce566fb878ad admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:30.813169 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ee54a-a418-48f6-95df-d06be013b2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:30.813795 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ee54a-a418-48f6-95df-d06be013b2ef admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:30.814079 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ee54a-a418-48f6-95df-d06be013b2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:30.814357 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193ee54a-a418-48f6-95df-d06be013b2ef admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:30.815092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ee54a-a418-48f6-95df-d06be013b2ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:30.817098 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193ee54a-a418-48f6-95df-d06be013b2ef admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:30.903694 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1092/2186] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:56:30 2022] POST /identity/v3/users => generated 293 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:56:30.907873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25421ebc-a38f-4af0-8764-f06b58a39267 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:30.918923 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25421ebc-a38f-4af0-8764-f06b58a39267 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:30.923882 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81d755-b304-4ea0-b70a-84f85c3b7fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:30.924502 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81d755-b304-4ea0-b70a-84f85c3b7fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:30.924791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81d755-b304-4ea0-b70a-84f85c3b7fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:30.925069 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81d755-b304-4ea0-b70a-84f85c3b7fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:30.925770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da81d755-b304-4ea0-b70a-84f85c3b7fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:30.927569 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da81d755-b304-4ea0-b70a-84f85c3b7fc8 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:30.938349 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1095/2187] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:30 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:56:30.942738 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ee54a-a418-48f6-95df-d06be013b2ef admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:30.954120 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193ee54a-a418-48f6-95df-d06be013b2ef admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:30.959284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb80ea6-e3d8-40d4-8282-17fa26cc829e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:30.959932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb80ea6-e3d8-40d4-8282-17fa26cc829e admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:30.960212 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb80ea6-e3d8-40d4-8282-17fa26cc829e admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:30.960486 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb80ea6-e3d8-40d4-8282-17fa26cc829e admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:30.961294 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb80ea6-e3d8-40d4-8282-17fa26cc829e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:30.962885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb80ea6-e3d8-40d4-8282-17fa26cc829e admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:30.978343 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1093/2188] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:30 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:56:30.979609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81d755-b304-4ea0-b70a-84f85c3b7fc8 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:30.990358 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81d755-b304-4ea0-b70a-84f85c3b7fc8 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:30.995158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c061c-85e6-417d-b866-6dac7030d89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:31.000072 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c061c-85e6-417d-b866-6dac7030d89a admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:31.000072 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c061c-85e6-417d-b866-6dac7030d89a admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:31.000072 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807c061c-85e6-417d-b866-6dac7030d89a admin admin] PATH_INFO: `/v3/projects/015ede30de72423bbb50126a729b559a/users/4a8bda9441f54554a38b65ccdafbff34/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:31.013974 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c061c-85e6-417d-b866-6dac7030d89a admin admin] RBAC: Authorizing `identity:create_grant(project_id=015ede30de72423bbb50126a729b559a, user_id=4a8bda9441f54554a38b65ccdafbff34, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:31.017333 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807c061c-85e6-417d-b866-6dac7030d89a admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:31.075964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1096/2189] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:56:30 2022] PUT /identity/v3/projects/015ede30de72423bbb50126a729b559a/users/4a8bda9441f54554a38b65ccdafbff34/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:31.087099 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c03404-1248-4f12-b98d-19eb63b34df2 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:31.087388 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c03404-1248-4f12-b98d-19eb63b34df2 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:31.087691 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c03404-1248-4f12-b98d-19eb63b34df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:31.116018 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-51c03404-1248-4f12-b98d-19eb63b34df2 None None] MFA Rules not processed for user `4a8bda9441f54554a38b65ccdafbff34`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:31.117264 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-51c03404-1248-4f12-b98d-19eb63b34df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:31.149815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:31.185045 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1094/2190] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:56:31.200783 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c061c-85e6-417d-b866-6dac7030d89a admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:31.200783 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-807c061c-85e6-417d-b866-6dac7030d89a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:31.266494 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807c061c-85e6-417d-b866-6dac7030d89a admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:31.270263 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83529571-4e7c-4cbd-a1d6-796bc6dbfa59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:31.270881 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83529571-4e7c-4cbd-a1d6-796bc6dbfa59 service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:31.271193 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83529571-4e7c-4cbd-a1d6-796bc6dbfa59 service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:31.271468 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83529571-4e7c-4cbd-a1d6-796bc6dbfa59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:31.280034 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:31.280210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83529571-4e7c-4cbd-a1d6-796bc6dbfa59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:31.281641 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83529571-4e7c-4cbd-a1d6-796bc6dbfa59 service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:31.291203 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1097/2191] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:56:31 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:31.311904 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c03404-1248-4f12-b98d-19eb63b34df2 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:31.379315 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c03404-1248-4f12-b98d-19eb63b34df2 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:31.384080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20189a3-c7bc-468e-a969-5db42b2a1167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:31.384742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20189a3-c7bc-468e-a969-5db42b2a1167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:31.385044 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20189a3-c7bc-468e-a969-5db42b2a1167 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:31.385321 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20189a3-c7bc-468e-a969-5db42b2a1167 admin admin] PATH_INFO: `/v3/users/4a8bda9441f54554a38b65ccdafbff34` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:31.387345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20189a3-c7bc-468e-a969-5db42b2a1167 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a8bda9441f54554a38b65ccdafbff34)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:31.392531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20189a3-c7bc-468e-a969-5db42b2a1167 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:31.601861 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-a20189a3-c7bc-468e-a969-5db42b2a1167 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a8bda9441f54554a38b65ccdafbff34'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:31.609079 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-a20189a3-c7bc-468e-a969-5db42b2a1167 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a8bda9441f54554a38b65ccdafbff34'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:31.609779 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-a20189a3-c7bc-468e-a969-5db42b2a1167 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a8bda9441f54554a38b65ccdafbff34'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:31.706919 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-a20189a3-c7bc-468e-a969-5db42b2a1167 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a8bda9441f54554a38b65ccdafbff34'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:31.721491 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-a20189a3-c7bc-468e-a969-5db42b2a1167 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a8bda9441f54554a38b65ccdafbff34'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:31.727938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1095/2192] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:56:31 2022] DELETE /identity/v3/users/4a8bda9441f54554a38b65ccdafbff34 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:31.732880 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde2c92-0833-4106-8a25-3ea338963fa1 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:31.733190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde2c92-0833-4106-8a25-3ea338963fa1 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:31.733467 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcde2c92-0833-4106-8a25-3ea338963fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:31.752586 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-bcde2c92-0833-4106-8a25-3ea338963fa1 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:31.753798 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-bcde2c92-0833-4106-8a25-3ea338963fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:31.764513 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:31.800218 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:31.823058 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1098/2193] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:31 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:31.833351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20189a3-c7bc-468e-a969-5db42b2a1167 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:31.834792 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-a20189a3-c7bc-468e-a969-5db42b2a1167 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:31.893464 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20189a3-c7bc-468e-a969-5db42b2a1167 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:31.896993 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f3e739-06b4-4da7-b99b-e490c5263b40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:31.897649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3e739-06b4-4da7-b99b-e490c5263b40 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:31.897929 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3e739-06b4-4da7-b99b-e490c5263b40 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:31.898202 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3e739-06b4-4da7-b99b-e490c5263b40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:31.908408 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f3e739-06b4-4da7-b99b-e490c5263b40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:31.909851 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f3e739-06b4-4da7-b99b-e490c5263b40 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:31.923418 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1096/2194] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:56:31 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:32.273850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:32.379195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde2c92-0833-4106-8a25-3ea338963fa1 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:32.380773 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-bcde2c92-0833-4106-8a25-3ea338963fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:32.393925 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcde2c92-0833-4106-8a25-3ea338963fa1 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:32.397363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:32.400802 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83b126f-a51d-4863-a5ca-573b67950822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:32.401557 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b126f-a51d-4863-a5ca-573b67950822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:32.401926 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b126f-a51d-4863-a5ca-573b67950822 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:32.402306 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83b126f-a51d-4863-a5ca-573b67950822 admin admin] PATH_INFO: `/v3/projects/015ede30de72423bbb50126a729b559a` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:32.403782 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83b126f-a51d-4863-a5ca-573b67950822 admin admin] RBAC: Authorizing `identity:delete_project(project_id=015ede30de72423bbb50126a729b559a)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:32.405755 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83b126f-a51d-4863-a5ca-573b67950822 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:32.523894 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e83b126f-a51d-4863-a5ca-573b67950822 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '015ede30de72423bbb50126a729b559a', 'request_id': 'req-e83b126f-a51d-4863-a5ca-573b67950822'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:32.531247 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e83b126f-a51d-4863-a5ca-573b67950822 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '015ede30de72423bbb50126a729b559a', 'request_id': 'req-e83b126f-a51d-4863-a5ca-573b67950822'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:32.628306 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e83b126f-a51d-4863-a5ca-573b67950822 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '015ede30de72423bbb50126a729b559a', 'request_id': 'req-e83b126f-a51d-4863-a5ca-573b67950822'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:32.640100 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e83b126f-a51d-4863-a5ca-573b67950822 admin admin] The token cache is being invalidate because project 015ede30de72423bbb50126a729b559a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:56:32.640385 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e83b126f-a51d-4863-a5ca-573b67950822 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:32.641017 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-e83b126f-a51d-4863-a5ca-573b67950822 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:32.642505 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1099/2195] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:56:32 2022] DELETE /identity/v3/projects/015ede30de72423bbb50126a729b559a => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:32.647584 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7b8bc-e634-4fb7-a2be-2127688609b1 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:32.648050 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7b8bc-e634-4fb7-a2be-2127688609b1 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:32.648050 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df7b8bc-e634-4fb7-a2be-2127688609b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:32.668987 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-9df7b8bc-e634-4fb7-a2be-2127688609b1 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:32.670231 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-9df7b8bc-e634-4fb7-a2be-2127688609b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:32.729801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1097/2196] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:32 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:32.737734 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23081d06-2717-432b-984e-9f70e726efd5 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:32.738056 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23081d06-2717-432b-984e-9f70e726efd5 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:32.738336 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23081d06-2717-432b-984e-9f70e726efd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:32.759220 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-23081d06-2717-432b-984e-9f70e726efd5 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:32.760504 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-23081d06-2717-432b-984e-9f70e726efd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:32.769232 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1100/2197] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:32 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:32.773632 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7b8bc-e634-4fb7-a2be-2127688609b1 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:32.783964 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df7b8bc-e634-4fb7-a2be-2127688609b1 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:32.789073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1963479-a481-4431-a0cb-20dec4bf2247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:32.789716 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1963479-a481-4431-a0cb-20dec4bf2247 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:32.790029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1963479-a481-4431-a0cb-20dec4bf2247 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:32.790305 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1963479-a481-4431-a0cb-20dec4bf2247 admin admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:32.791005 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1963479-a481-4431-a0cb-20dec4bf2247 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:56:32.791347 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1963479-a481-4431-a0cb-20dec4bf2247 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:32.793144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1963479-a481-4431-a0cb-20dec4bf2247 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:32.804986 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1098/2198] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:56:32 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:56:32.810036 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23081d06-2717-432b-984e-9f70e726efd5 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:32.819553 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23081d06-2717-432b-984e-9f70e726efd5 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:32.824934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42b9e3-08d9-4b14-b579-3763aa85f574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:32.825542 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42b9e3-08d9-4b14-b579-3763aa85f574 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:32.825823 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42b9e3-08d9-4b14-b579-3763aa85f574 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:32.826101 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d42b9e3-08d9-4b14-b579-3763aa85f574 admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:32.826817 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d42b9e3-08d9-4b14-b579-3763aa85f574 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:32.828785 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d42b9e3-08d9-4b14-b579-3763aa85f574 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:32.932726 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1101/2199] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:56:32 2022] POST /identity/v3/projects => generated 372 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:56:32.937669 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1963479-a481-4431-a0cb-20dec4bf2247 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:32.947627 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1963479-a481-4431-a0cb-20dec4bf2247 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:32.999189 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:33.011506 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf624c-d1ec-44df-b055-72391a0c9953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:33.012223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf624c-d1ec-44df-b055-72391a0c9953 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:33.012504 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf624c-d1ec-44df-b055-72391a0c9953 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:33.012776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baf624c-d1ec-44df-b055-72391a0c9953 admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:33.013507 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf624c-d1ec-44df-b055-72391a0c9953 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:33.015744 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baf624c-d1ec-44df-b055-72391a0c9953 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:33.093537 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1099/2200] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:56:32 2022] POST /identity/v3/users => generated 287 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:56:33.094968 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42b9e3-08d9-4b14-b579-3763aa85f574 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:33.108837 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d42b9e3-08d9-4b14-b579-3763aa85f574 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:33.112437 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739358f-5d9b-4b68-b01a-328242748ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:33.113083 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739358f-5d9b-4b68-b01a-328242748ec7 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:33.113364 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739358f-5d9b-4b68-b01a-328242748ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:33.113644 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3739358f-5d9b-4b68-b01a-328242748ec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:33.114347 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3739358f-5d9b-4b68-b01a-328242748ec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:33.115974 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3739358f-5d9b-4b68-b01a-328242748ec7 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:33.127207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1102/2201] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:33 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:56:33.132302 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf624c-d1ec-44df-b055-72391a0c9953 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:33.143189 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baf624c-d1ec-44df-b055-72391a0c9953 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:33.147935 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bf7d9f-c86c-474f-936a-32e7af168724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:33.148549 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf7d9f-c86c-474f-936a-32e7af168724 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:33.148831 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf7d9f-c86c-474f-936a-32e7af168724 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:33.149108 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bf7d9f-c86c-474f-936a-32e7af168724 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:33.149799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf7d9f-c86c-474f-936a-32e7af168724 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:33.151431 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bf7d9f-c86c-474f-936a-32e7af168724 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:33.163083 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1100/2202] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:33 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:56:33.168049 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739358f-5d9b-4b68-b01a-328242748ec7 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:33.178350 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3739358f-5d9b-4b68-b01a-328242748ec7 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:33.184087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e495cb4-2f6d-45df-bbd2-361ad12a07c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:33.184888 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e495cb4-2f6d-45df-bbd2-361ad12a07c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:33.185175 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e495cb4-2f6d-45df-bbd2-361ad12a07c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:33.185471 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e495cb4-2f6d-45df-bbd2-361ad12a07c5 admin admin] PATH_INFO: `/v3/projects/5291f7a35f644effb7591c92c732823a/users/52952f60728549f6ba8ee28effe0c877/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:33.201808 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e495cb4-2f6d-45df-bbd2-361ad12a07c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5291f7a35f644effb7591c92c732823a, user_id=52952f60728549f6ba8ee28effe0c877, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:33.204606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e495cb4-2f6d-45df-bbd2-361ad12a07c5 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:33.309536 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1103/2203] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:56:33 2022] PUT /identity/v3/projects/5291f7a35f644effb7591c92c732823a/users/52952f60728549f6ba8ee28effe0c877/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:33.319969 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986c844-f16e-4121-8fc9-9194289dc38b None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:33.320252 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986c844-f16e-4121-8fc9-9194289dc38b None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:33.320526 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d986c844-f16e-4121-8fc9-9194289dc38b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:33.350778 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-d986c844-f16e-4121-8fc9-9194289dc38b None None] MFA Rules not processed for user `52952f60728549f6ba8ee28effe0c877`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:33.352363 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-d986c844-f16e-4121-8fc9-9194289dc38b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:33.421862 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1101/2204] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:33.439327 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e495cb4-2f6d-45df-bbd2-361ad12a07c5 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:33.441114 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-4e495cb4-2f6d-45df-bbd2-361ad12a07c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:33.461187 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:33.500990 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e495cb4-2f6d-45df-bbd2-361ad12a07c5 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:33.504758 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd68a11-4db9-4b62-94a5-2f2673ccd098 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:33.505403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd68a11-4db9-4b62-94a5-2f2673ccd098 service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:33.505677 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd68a11-4db9-4b62-94a5-2f2673ccd098 service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:33.505951 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd68a11-4db9-4b62-94a5-2f2673ccd098 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:33.514646 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd68a11-4db9-4b62-94a5-2f2673ccd098 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:33.516207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd68a11-4db9-4b62-94a5-2f2673ccd098 service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:33.524952 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1104/2205] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:56:33 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:33.552141 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986c844-f16e-4121-8fc9-9194289dc38b None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:33.620508 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d986c844-f16e-4121-8fc9-9194289dc38b None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:33.625265 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9facf-29af-4bfe-bc6b-f38dc74d1cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:33.625902 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9facf-29af-4bfe-bc6b-f38dc74d1cb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:33.626180 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9facf-29af-4bfe-bc6b-f38dc74d1cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:33.626450 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9facf-29af-4bfe-bc6b-f38dc74d1cb6 admin admin] PATH_INFO: `/v3/users/52952f60728549f6ba8ee28effe0c877` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:33.627673 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee9facf-29af-4bfe-bc6b-f38dc74d1cb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52952f60728549f6ba8ee28effe0c877)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:33.629640 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee9facf-29af-4bfe-bc6b-f38dc74d1cb6 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:33.854072 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-0ee9facf-29af-4bfe-bc6b-f38dc74d1cb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52952f60728549f6ba8ee28effe0c877'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:33.861504 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-0ee9facf-29af-4bfe-bc6b-f38dc74d1cb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52952f60728549f6ba8ee28effe0c877'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:33.862199 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-0ee9facf-29af-4bfe-bc6b-f38dc74d1cb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52952f60728549f6ba8ee28effe0c877'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:33.865272 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:33.965538 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-0ee9facf-29af-4bfe-bc6b-f38dc74d1cb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52952f60728549f6ba8ee28effe0c877'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:33.979095 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-0ee9facf-29af-4bfe-bc6b-f38dc74d1cb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52952f60728549f6ba8ee28effe0c877'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:33.985829 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1102/2206] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:56:33 2022] DELETE /identity/v3/users/52952f60728549f6ba8ee28effe0c877 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:33.990197 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9ec5d-49db-4a83-a402-3cf20f74baa2 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:33.990488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9ec5d-49db-4a83-a402-3cf20f74baa2 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:33.990767 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f9ec5d-49db-4a83-a402-3cf20f74baa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:34.011656 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-19f9ec5d-49db-4a83-a402-3cf20f74baa2 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:34.012981 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-19f9ec5d-49db-4a83-a402-3cf20f74baa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:34.072385 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1105/2207] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:33 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:34.082817 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9facf-29af-4bfe-bc6b-f38dc74d1cb6 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:34.084310 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-0ee9facf-29af-4bfe-bc6b-f38dc74d1cb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:34.141487 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9facf-29af-4bfe-bc6b-f38dc74d1cb6 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:34.145459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f3e090-e88d-4882-8e39-1c6dff209ad8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:34.146103 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3e090-e88d-4882-8e39-1c6dff209ad8 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:34.146441 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3e090-e88d-4882-8e39-1c6dff209ad8 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:34.146720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3e090-e88d-4882-8e39-1c6dff209ad8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:34.156381 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f3e090-e88d-4882-8e39-1c6dff209ad8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:34.157886 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f3e090-e88d-4882-8e39-1c6dff209ad8 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:34.169958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1103/2208] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:56:34 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:34.720897 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9ec5d-49db-4a83-a402-3cf20f74baa2 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:34.722339 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-19f9ec5d-49db-4a83-a402-3cf20f74baa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:34.736637 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f9ec5d-49db-4a83-a402-3cf20f74baa2 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:34.741404 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec01604d-f7cc-4a06-9959-c8e473bd70b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:34.742052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01604d-f7cc-4a06-9959-c8e473bd70b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:34.742329 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01604d-f7cc-4a06-9959-c8e473bd70b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:34.742604 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec01604d-f7cc-4a06-9959-c8e473bd70b8 admin admin] PATH_INFO: `/v3/projects/5291f7a35f644effb7591c92c732823a` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:34.744305 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec01604d-f7cc-4a06-9959-c8e473bd70b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5291f7a35f644effb7591c92c732823a)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:34.746848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec01604d-f7cc-4a06-9959-c8e473bd70b8 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:34.846727 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-ec01604d-f7cc-4a06-9959-c8e473bd70b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5291f7a35f644effb7591c92c732823a', 'request_id': 'req-ec01604d-f7cc-4a06-9959-c8e473bd70b8'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:34.853103 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-ec01604d-f7cc-4a06-9959-c8e473bd70b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5291f7a35f644effb7591c92c732823a', 'request_id': 'req-ec01604d-f7cc-4a06-9959-c8e473bd70b8'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:34.930596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-ec01604d-f7cc-4a06-9959-c8e473bd70b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5291f7a35f644effb7591c92c732823a', 'request_id': 'req-ec01604d-f7cc-4a06-9959-c8e473bd70b8'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:34.943144 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-ec01604d-f7cc-4a06-9959-c8e473bd70b8 admin admin] The token cache is being invalidate because project 5291f7a35f644effb7591c92c732823a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:56:34.943435 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-ec01604d-f7cc-4a06-9959-c8e473bd70b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:34.944106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-ec01604d-f7cc-4a06-9959-c8e473bd70b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:34.945748 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1106/2209] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:56:34 2022] DELETE /identity/v3/projects/5291f7a35f644effb7591c92c732823a => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:34.951294 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849629f-bb5c-4b39-a827-2149fcef3ac8 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:34.951805 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849629f-bb5c-4b39-a827-2149fcef3ac8 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:34.951915 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849629f-bb5c-4b39-a827-2149fcef3ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:34.971946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-f849629f-bb5c-4b39-a827-2149fcef3ac8 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:34.973210 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f849629f-bb5c-4b39-a827-2149fcef3ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:35.035450 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1104/2210] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:34 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:35.045586 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c6688-18d6-4c44-9f06-f8f8508ea624 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:35.045930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c6688-18d6-4c44-9f06-f8f8508ea624 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:35.046189 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126c6688-18d6-4c44-9f06-f8f8508ea624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:35.066854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-126c6688-18d6-4c44-9f06-f8f8508ea624 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:35.068111 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-126c6688-18d6-4c44-9f06-f8f8508ea624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:35.076164 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1107/2211] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:35.080348 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849629f-bb5c-4b39-a827-2149fcef3ac8 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:35.091238 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849629f-bb5c-4b39-a827-2149fcef3ac8 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:35.096065 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69380f1c-bd09-4c15-86f7-57107efd6487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:35.096682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69380f1c-bd09-4c15-86f7-57107efd6487 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:35.096960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69380f1c-bd09-4c15-86f7-57107efd6487 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:35.097234 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69380f1c-bd09-4c15-86f7-57107efd6487 admin admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:35.097924 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69380f1c-bd09-4c15-86f7-57107efd6487 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:56:35.098241 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69380f1c-bd09-4c15-86f7-57107efd6487 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:35.099903 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69380f1c-bd09-4c15-86f7-57107efd6487 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:35.112367 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1105/2212] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:56:35 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:56:35.117215 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c6688-18d6-4c44-9f06-f8f8508ea624 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:35.126974 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126c6688-18d6-4c44-9f06-f8f8508ea624 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:35.132134 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67edb564-2314-4273-b853-d2e234f05c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:35.132749 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edb564-2314-4273-b853-d2e234f05c77 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:35.133038 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edb564-2314-4273-b853-d2e234f05c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:35.133314 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67edb564-2314-4273-b853-d2e234f05c77 admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:35.133993 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67edb564-2314-4273-b853-d2e234f05c77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:35.136046 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67edb564-2314-4273-b853-d2e234f05c77 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:35.212178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1108/2213] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:56:35 2022] POST /identity/v3/projects => generated 380 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:56:35.217228 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69380f1c-bd09-4c15-86f7-57107efd6487 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:35.227328 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69380f1c-bd09-4c15-86f7-57107efd6487 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:35.286575 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbb580-fef7-4be1-8cf7-3c7da7e392ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:35.287144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbb580-fef7-4be1-8cf7-3c7da7e392ee admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:35.287424 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbb580-fef7-4be1-8cf7-3c7da7e392ee admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:35.287696 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fbb580-fef7-4be1-8cf7-3c7da7e392ee admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:35.288370 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fbb580-fef7-4be1-8cf7-3c7da7e392ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:35.290336 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fbb580-fef7-4be1-8cf7-3c7da7e392ee admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:35.302470 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:35.414542 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1106/2214] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:56:35 2022] POST /identity/v3/users => generated 291 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:56:35.416114 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67edb564-2314-4273-b853-d2e234f05c77 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:35.427333 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67edb564-2314-4273-b853-d2e234f05c77 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:35.440105 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afed9ef-9d1b-432f-af46-8273d36b1029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:35.441010 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afed9ef-9d1b-432f-af46-8273d36b1029 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:35.441406 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afed9ef-9d1b-432f-af46-8273d36b1029 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:35.441770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afed9ef-9d1b-432f-af46-8273d36b1029 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:35.442699 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afed9ef-9d1b-432f-af46-8273d36b1029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:35.444919 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afed9ef-9d1b-432f-af46-8273d36b1029 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:35.460481 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1109/2215] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:35 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:56:35.461651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbb580-fef7-4be1-8cf7-3c7da7e392ee admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:35.472668 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fbb580-fef7-4be1-8cf7-3c7da7e392ee admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:35.478906 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1bec66-0bb4-4ebf-862a-c09d0b250df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:35.479502 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1bec66-0bb4-4ebf-862a-c09d0b250df9 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:35.480351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1bec66-0bb4-4ebf-862a-c09d0b250df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:35.480647 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1bec66-0bb4-4ebf-862a-c09d0b250df9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:35.481351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1bec66-0bb4-4ebf-862a-c09d0b250df9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:35.483035 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1bec66-0bb4-4ebf-862a-c09d0b250df9 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:35.496441 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1107/2216] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:35 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:56:35.500375 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afed9ef-9d1b-432f-af46-8273d36b1029 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:35.505870 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:35.511343 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afed9ef-9d1b-432f-af46-8273d36b1029 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:35.516049 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195d3a4-9b83-4645-af6a-cb3bf6580454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:35.516721 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195d3a4-9b83-4645-af6a-cb3bf6580454 admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:35.517003 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195d3a4-9b83-4645-af6a-cb3bf6580454 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:35.517279 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195d3a4-9b83-4645-af6a-cb3bf6580454 admin admin] PATH_INFO: `/v3/projects/be66234794284419a47e3afcae968084/users/914de37664914b4cb91e0ce2c3a0e1d1/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:35.532093 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4195d3a4-9b83-4645-af6a-cb3bf6580454 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be66234794284419a47e3afcae968084, user_id=914de37664914b4cb91e0ce2c3a0e1d1, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:35.534840 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4195d3a4-9b83-4645-af6a-cb3bf6580454 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:35.652357 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1110/2217] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:56:35 2022] PUT /identity/v3/projects/be66234794284419a47e3afcae968084/users/914de37664914b4cb91e0ce2c3a0e1d1/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:35.657424 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03579e6-0b3c-4f62-aaeb-db927c431b73 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:35.657708 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03579e6-0b3c-4f62-aaeb-db927c431b73 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:35.657982 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03579e6-0b3c-4f62-aaeb-db927c431b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:35.683851 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-f03579e6-0b3c-4f62-aaeb-db927c431b73 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:35.685240 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f03579e6-0b3c-4f62-aaeb-db927c431b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:35.766094 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1108/2218] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:35.783516 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195d3a4-9b83-4645-af6a-cb3bf6580454 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:35.788827 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-4195d3a4-9b83-4645-af6a-cb3bf6580454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:36.023228 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195d3a4-9b83-4645-af6a-cb3bf6580454 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:36.023228 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f4866-81cd-40d1-a236-88133511054f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:36.023228 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f4866-81cd-40d1-a236-88133511054f service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:36.023228 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f4866-81cd-40d1-a236-88133511054f service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:36.024827 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2f4866-81cd-40d1-a236-88133511054f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:36.024827 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2f4866-81cd-40d1-a236-88133511054f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:36.024827 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2f4866-81cd-40d1-a236-88133511054f service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:36.024827 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1111/2219] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:56:35 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:42.770146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0c027-8e4c-45d0-8ccd-b1ff4ada7aa4 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:42.770692 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0c027-8e4c-45d0-8ccd-b1ff4ada7aa4 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:42.770692 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0c027-8e4c-45d0-8ccd-b1ff4ada7aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:42.803374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-6ad0c027-8e4c-45d0-8ccd-b1ff4ada7aa4 None None] MFA Rules not processed for user `914de37664914b4cb91e0ce2c3a0e1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:42.804981 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-6ad0c027-8e4c-45d0-8ccd-b1ff4ada7aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:42.873640 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1109/2220] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:42 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:42.878114 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f4866-81cd-40d1-a236-88133511054f service neutron] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:42.888253 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2f4866-81cd-40d1-a236-88133511054f service neutron] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:42.892950 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe044bb-291e-4081-a822-d6b85337dc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:42.893505 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe044bb-291e-4081-a822-d6b85337dc4a admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:42.893992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe044bb-291e-4081-a822-d6b85337dc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:42.894238 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe044bb-291e-4081-a822-d6b85337dc4a admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:42.894911 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe044bb-291e-4081-a822-d6b85337dc4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:42.896852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe044bb-291e-4081-a822-d6b85337dc4a admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:43.000380 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1112/2221] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:56:42 2022] POST /identity/v3/projects => generated 378 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:56:43.008661 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0c027-8e4c-45d0-8ccd-b1ff4ada7aa4 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:43.020797 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad0c027-8e4c-45d0-8ccd-b1ff4ada7aa4 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:43.084525 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c33df8-ecb9-424c-86af-b40c710fd4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:43.086419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c33df8-ecb9-424c-86af-b40c710fd4ff admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:43.086419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c33df8-ecb9-424c-86af-b40c710fd4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:43.086419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c33df8-ecb9-424c-86af-b40c710fd4ff admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:43.086419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c33df8-ecb9-424c-86af-b40c710fd4ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:43.088304 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c33df8-ecb9-424c-86af-b40c710fd4ff admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:43.205559 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1110/2222] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:56:43 2022] POST /identity/v3/users => generated 296 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:56:43.209751 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe044bb-291e-4081-a822-d6b85337dc4a admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:43.221197 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe044bb-291e-4081-a822-d6b85337dc4a admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:43.225859 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b141d4-1a2b-4e8f-aa74-a4831c023af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:43.226472 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b141d4-1a2b-4e8f-aa74-a4831c023af8 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:43.226753 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b141d4-1a2b-4e8f-aa74-a4831c023af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:43.227028 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b141d4-1a2b-4e8f-aa74-a4831c023af8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:43.227775 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b141d4-1a2b-4e8f-aa74-a4831c023af8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:43.229377 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b141d4-1a2b-4e8f-aa74-a4831c023af8 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:43.241227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1113/2223] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:43 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:56:43.245549 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c33df8-ecb9-424c-86af-b40c710fd4ff admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:43.252317 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:43.256068 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c33df8-ecb9-424c-86af-b40c710fd4ff admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:43.260969 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a1551-1584-4d37-b317-3d1a23adca2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:43.261619 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a1551-1584-4d37-b317-3d1a23adca2d admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:43.261899 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a1551-1584-4d37-b317-3d1a23adca2d admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:43.262170 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a1551-1584-4d37-b317-3d1a23adca2d admin admin] PATH_INFO: `/v3/projects/07312de08a584c85a9e20a1da10d58ed/users/95c028783b47498f817dbfc51e7958a0/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:43.279444 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a1551-1584-4d37-b317-3d1a23adca2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=07312de08a584c85a9e20a1da10d58ed, user_id=95c028783b47498f817dbfc51e7958a0, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:43.282095 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a1551-1584-4d37-b317-3d1a23adca2d admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:43.412231 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1111/2224] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:56:43 2022] PUT /identity/v3/projects/07312de08a584c85a9e20a1da10d58ed/users/95c028783b47498f817dbfc51e7958a0/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:43.416501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b141d4-1a2b-4e8f-aa74-a4831c023af8 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:43.427625 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b141d4-1a2b-4e8f-aa74-a4831c023af8 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:43.503774 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7c248-9675-4c44-93f2-5ede8ec8f625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:43.504411 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7c248-9675-4c44-93f2-5ede8ec8f625 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:43.504686 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7c248-9675-4c44-93f2-5ede8ec8f625 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:43.504956 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7c248-9675-4c44-93f2-5ede8ec8f625 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:43.505653 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7c248-9675-4c44-93f2-5ede8ec8f625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:43.507360 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7c248-9675-4c44-93f2-5ede8ec8f625 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:43.521690 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1114/2225] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:43 2022] GET /identity/v3/roles => generated 1427 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:56:43.523261 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a1551-1584-4d37-b317-3d1a23adca2d admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:43.536824 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a1551-1584-4d37-b317-3d1a23adca2d admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:43.541282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b1090-455b-4514-9f2f-e60fb003752e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:43.542403 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b1090-455b-4514-9f2f-e60fb003752e admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:43.542403 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b1090-455b-4514-9f2f-e60fb003752e admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:43.542644 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8b1090-455b-4514-9f2f-e60fb003752e admin admin] PATH_INFO: `/v3/projects/07312de08a584c85a9e20a1da10d58ed/users/95c028783b47498f817dbfc51e7958a0/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:43.545088 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b1090-455b-4514-9f2f-e60fb003752e admin admin] RBAC: Authorizing `identity:create_grant(project_id=07312de08a584c85a9e20a1da10d58ed, user_id=95c028783b47498f817dbfc51e7958a0, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:43.547938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8b1090-455b-4514-9f2f-e60fb003752e admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:43.559365 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1112/2226] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:56:43 2022] PUT /identity/v3/projects/07312de08a584c85a9e20a1da10d58ed/users/95c028783b47498f817dbfc51e7958a0/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:51.061614 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714caa6d-d553-4ab4-9a21-c40af9677fd5 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:51.062185 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714caa6d-d553-4ab4-9a21-c40af9677fd5 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:51.062278 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714caa6d-d553-4ab4-9a21-c40af9677fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:51.093334 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-714caa6d-d553-4ab4-9a21-c40af9677fd5 None None] MFA Rules not processed for user `95c028783b47498f817dbfc51e7958a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:51.094563 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-714caa6d-d553-4ab4-9a21-c40af9677fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:51.130044 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:51.175637 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1115/2227] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:51 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:51.183647 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b1090-455b-4514-9f2f-e60fb003752e admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:51.185712 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-7c8b1090-455b-4514-9f2f-e60fb003752e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:51.276062 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8b1090-455b-4514-9f2f-e60fb003752e admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:51.283249 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c539010d-0f95-422c-8c4c-8218182738e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:51.283827 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539010d-0f95-422c-8c4c-8218182738e0 service nova] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:51.284131 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539010d-0f95-422c-8c4c-8218182738e0 service nova] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:51.284414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c539010d-0f95-422c-8c4c-8218182738e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:51.342831 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c539010d-0f95-422c-8c4c-8218182738e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:51.344471 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c539010d-0f95-422c-8c4c-8218182738e0 service nova] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:51.353875 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1113/2228] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:56:51 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:52.592551 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805eb70-3c09-4f5a-992a-60333e6168f6 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:52.593003 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805eb70-3c09-4f5a-992a-60333e6168f6 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:52.593119 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6805eb70-3c09-4f5a-992a-60333e6168f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:52.618522 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-6805eb70-3c09-4f5a-992a-60333e6168f6 None None] MFA Rules not processed for user `53bc14ada1564911b2f0a698aef3f333`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:52.622622 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-6805eb70-3c09-4f5a-992a-60333e6168f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:52.690327 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1116/2229] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:52 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:57.333007 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c539010d-0f95-422c-8c4c-8218182738e0 service nova] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:57.335271 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-c539010d-0f95-422c-8c4c-8218182738e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:57.404450 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c539010d-0f95-422c-8c4c-8218182738e0 service nova] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:57.408956 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a9d6c-ff0f-43e8-a61f-fcfd5a531d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:57.409573 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a9d6c-ff0f-43e8-a61f-fcfd5a531d2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:57.409850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a9d6c-ff0f-43e8-a61f-fcfd5a531d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:57.410122 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a9d6c-ff0f-43e8-a61f-fcfd5a531d2f admin admin] PATH_INFO: `/v3/users/868368a034534937b0c795e929a861bb` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:57.411566 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01a9d6c-ff0f-43e8-a61f-fcfd5a531d2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=868368a034534937b0c795e929a861bb)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:57.413489 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01a9d6c-ff0f-43e8-a61f-fcfd5a531d2f admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:57.418118 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:57.648913 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-d01a9d6c-ff0f-43e8-a61f-fcfd5a531d2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '868368a034534937b0c795e929a861bb'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:57.659305 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-d01a9d6c-ff0f-43e8-a61f-fcfd5a531d2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '868368a034534937b0c795e929a861bb'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:57.660644 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-d01a9d6c-ff0f-43e8-a61f-fcfd5a531d2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '868368a034534937b0c795e929a861bb'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:57.837123 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-d01a9d6c-ff0f-43e8-a61f-fcfd5a531d2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '868368a034534937b0c795e929a861bb'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:57.854160 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-d01a9d6c-ff0f-43e8-a61f-fcfd5a531d2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '868368a034534937b0c795e929a861bb'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:57.875132 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1114/2230] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:56:57 2022] DELETE /identity/v3/users/868368a034534937b0c795e929a861bb => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:56:57.881410 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee707ca2-4a89-47d4-acbf-fe23586c639f None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:57.881973 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee707ca2-4a89-47d4-acbf-fe23586c639f None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:57.882101 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee707ca2-4a89-47d4-acbf-fe23586c639f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:57.911740 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-ee707ca2-4a89-47d4-acbf-fe23586c639f None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:57.913215 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-ee707ca2-4a89-47d4-acbf-fe23586c639f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:58.008542 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1117/2231] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:57 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:56:58.024914 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a9d6c-ff0f-43e8-a61f-fcfd5a531d2f admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:58.028491 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-d01a9d6c-ff0f-43e8-a61f-fcfd5a531d2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:58.103013 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a9d6c-ff0f-43e8-a61f-fcfd5a531d2f admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:58.109375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748cedf-bd23-462e-8de7-776fd17a18cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:58.110294 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748cedf-bd23-462e-8de7-776fd17a18cc service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:58.110765 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748cedf-bd23-462e-8de7-776fd17a18cc service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:58.112018 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748cedf-bd23-462e-8de7-776fd17a18cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:58.123635 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9748cedf-bd23-462e-8de7-776fd17a18cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:58.125322 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9748cedf-bd23-462e-8de7-776fd17a18cc service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:58.135901 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1115/2232] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:56:58 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:56:58.864397 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee707ca2-4a89-47d4-acbf-fe23586c639f None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:58.871731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-ee707ca2-4a89-47d4-acbf-fe23586c639f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:58.891413 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee707ca2-4a89-47d4-acbf-fe23586c639f None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:58.902722 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f1d9dc-1991-474d-869f-e914dd82cffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:58.902722 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1d9dc-1991-474d-869f-e914dd82cffe admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:58.902722 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1d9dc-1991-474d-869f-e914dd82cffe admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:58.902722 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f1d9dc-1991-474d-869f-e914dd82cffe admin admin] PATH_INFO: `/v3/projects/2e30e47b0d0744d7baa61f43e562ed0a` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:58.903857 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f1d9dc-1991-474d-869f-e914dd82cffe admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e30e47b0d0744d7baa61f43e562ed0a)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:58.903857 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f1d9dc-1991-474d-869f-e914dd82cffe admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:59.018745 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-75f1d9dc-1991-474d-869f-e914dd82cffe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e30e47b0d0744d7baa61f43e562ed0a', 'request_id': 'req-75f1d9dc-1991-474d-869f-e914dd82cffe'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:59.026224 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-75f1d9dc-1991-474d-869f-e914dd82cffe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e30e47b0d0744d7baa61f43e562ed0a', 'request_id': 'req-75f1d9dc-1991-474d-869f-e914dd82cffe'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:59.107741 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-75f1d9dc-1991-474d-869f-e914dd82cffe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e30e47b0d0744d7baa61f43e562ed0a', 'request_id': 'req-75f1d9dc-1991-474d-869f-e914dd82cffe'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:59.132839 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:59.139131 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-75f1d9dc-1991-474d-869f-e914dd82cffe admin admin] The token cache is being invalidate because project 2e30e47b0d0744d7baa61f43e562ed0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:56:59.139763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-75f1d9dc-1991-474d-869f-e914dd82cffe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:59.140645 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-75f1d9dc-1991-474d-869f-e914dd82cffe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:59.142696 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1118/2233] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:56:58 2022] DELETE /identity/v3/projects/2e30e47b0d0744d7baa61f43e562ed0a => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:59.147521 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bfcb1-d926-4b2d-92f1-292881790538 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:59.148487 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bfcb1-d926-4b2d-92f1-292881790538 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:59.148920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0bfcb1-d926-4b2d-92f1-292881790538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:59.170398 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-bf0bfcb1-d926-4b2d-92f1-292881790538 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:59.171667 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-bf0bfcb1-d926-4b2d-92f1-292881790538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:59.245241 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1116/2234] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:59 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:59.256151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ed7d8-bbb4-4900-99da-762d3771068d None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:59.256544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ed7d8-bbb4-4900-99da-762d3771068d None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:59.257398 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ed7d8-bbb4-4900-99da-762d3771068d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:59.283621 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-930ed7d8-bbb4-4900-99da-762d3771068d None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:59.284836 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-930ed7d8-bbb4-4900-99da-762d3771068d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:59.295249 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1119/2235] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:59 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:56:59.298130 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0bfcb1-d926-4b2d-92f1-292881790538 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:59.317613 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0bfcb1-d926-4b2d-92f1-292881790538 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:59.332156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8d41b-16bd-4ebb-8a04-ee08c13b7608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:59.336471 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8d41b-16bd-4ebb-8a04-ee08c13b7608 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:59.336471 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8d41b-16bd-4ebb-8a04-ee08c13b7608 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:59.336471 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c8d41b-16bd-4ebb-8a04-ee08c13b7608 admin admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:59.336471 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c8d41b-16bd-4ebb-8a04-ee08c13b7608 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:56:59.336471 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c8d41b-16bd-4ebb-8a04-ee08c13b7608 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:59.337810 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c8d41b-16bd-4ebb-8a04-ee08c13b7608 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:59.352067 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1117/2236] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:56:59 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 10:56:59.357527 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ed7d8-bbb4-4900-99da-762d3771068d None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:59.367144 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ed7d8-bbb4-4900-99da-762d3771068d None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:59.372378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c98dba-57bb-4e7d-a749-1208dea6bfe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:59.373640 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c98dba-57bb-4e7d-a749-1208dea6bfe6 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:59.373640 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c98dba-57bb-4e7d-a749-1208dea6bfe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:59.373640 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c98dba-57bb-4e7d-a749-1208dea6bfe6 admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:59.374316 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c98dba-57bb-4e7d-a749-1208dea6bfe6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:59.376374 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c98dba-57bb-4e7d-a749-1208dea6bfe6 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:59.463707 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1120/2237] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:56:59 2022] POST /identity/v3/projects => generated 364 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:56:59.468613 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8d41b-16bd-4ebb-8a04-ee08c13b7608 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:59.478671 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c8d41b-16bd-4ebb-8a04-ee08c13b7608 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:59.487629 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:56:59.536426 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41ad8e-8e0c-49af-aab0-64ef5a61f09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:59.537037 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41ad8e-8e0c-49af-aab0-64ef5a61f09d admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:59.537314 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41ad8e-8e0c-49af-aab0-64ef5a61f09d admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:59.537586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41ad8e-8e0c-49af-aab0-64ef5a61f09d admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:59.538265 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41ad8e-8e0c-49af-aab0-64ef5a61f09d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:59.540287 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d41ad8e-8e0c-49af-aab0-64ef5a61f09d admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:59.595233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c98dba-57bb-4e7d-a749-1208dea6bfe6 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:59.598102 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-f4c98dba-57bb-4e7d-a749-1208dea6bfe6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:59.614234 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c98dba-57bb-4e7d-a749-1208dea6bfe6 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:59.619092 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d3b1cf-430a-4a79-a587-5af3cf8198ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:59.619708 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3b1cf-430a-4a79-a587-5af3cf8198ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:59.620019 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3b1cf-430a-4a79-a587-5af3cf8198ca admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:59.620301 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3b1cf-430a-4a79-a587-5af3cf8198ca admin admin] PATH_INFO: `/v3/users/e0618aad2b69470c9f8607d2f5925eea` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:59.622059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d3b1cf-430a-4a79-a587-5af3cf8198ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0618aad2b69470c9f8607d2f5925eea)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:59.624052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d3b1cf-430a-4a79-a587-5af3cf8198ca admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:59.655419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1118/2238] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:56:59 2022] POST /identity/v3/users => generated 283 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:56:59.656550 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41ad8e-8e0c-49af-aab0-64ef5a61f09d admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:59.666920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41ad8e-8e0c-49af-aab0-64ef5a61f09d admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:59.672413 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492668b-7ba8-452e-9ad0-d62992b93a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:59.673188 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492668b-7ba8-452e-9ad0-d62992b93a81 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:59.673473 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492668b-7ba8-452e-9ad0-d62992b93a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:59.673749 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492668b-7ba8-452e-9ad0-d62992b93a81 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:59.674480 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492668b-7ba8-452e-9ad0-d62992b93a81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:59.676233 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7492668b-7ba8-452e-9ad0-d62992b93a81 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:59.693327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1119/2239] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:59 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:56:59.697705 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492668b-7ba8-452e-9ad0-d62992b93a81 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:59.709214 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492668b-7ba8-452e-9ad0-d62992b93a81 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:59.727678 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c723f-3fdf-49df-9288-d96f202d3f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:59.728347 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c723f-3fdf-49df-9288-d96f202d3f5f admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:59.728799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c723f-3fdf-49df-9288-d96f202d3f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:59.729066 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c723f-3fdf-49df-9288-d96f202d3f5f admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:59.730673 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c723f-3fdf-49df-9288-d96f202d3f5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:59.732410 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c723f-3fdf-49df-9288-d96f202d3f5f admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:59.745664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1120/2240] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:56:59 2022] GET /identity/v3/roles => generated 1427 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:56:59.751456 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c723f-3fdf-49df-9288-d96f202d3f5f admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:56:59.763714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c723f-3fdf-49df-9288-d96f202d3f5f admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:56:59.768256 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa6dd9c-b0ed-465a-9f3a-2e78ad126468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:56:59.768917 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6dd9c-b0ed-465a-9f3a-2e78ad126468 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:59.769198 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6dd9c-b0ed-465a-9f3a-2e78ad126468 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:59.772400 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa6dd9c-b0ed-465a-9f3a-2e78ad126468 admin admin] PATH_INFO: `/v3/projects/eccf25885e8f49d988e99f5ce9ad8d4a/users/8237f2e820ba4eaa9f730afac10288b9/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:59.785663 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa6dd9c-b0ed-465a-9f3a-2e78ad126468 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eccf25885e8f49d988e99f5ce9ad8d4a, user_id=8237f2e820ba4eaa9f730afac10288b9, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:56:59.788652 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa6dd9c-b0ed-465a-9f3a-2e78ad126468 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:56:59.830269 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-81d3b1cf-430a-4a79-a587-5af3cf8198ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0618aad2b69470c9f8607d2f5925eea'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:59.830964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-81d3b1cf-430a-4a79-a587-5af3cf8198ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0618aad2b69470c9f8607d2f5925eea'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:59.850220 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1121/2241] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:56:59 2022] PUT /identity/v3/projects/eccf25885e8f49d988e99f5ce9ad8d4a/users/8237f2e820ba4eaa9f730afac10288b9/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:56:59.861565 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482ac2c-49ce-4404-8afd-b3513a4568c2 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:59.861917 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482ac2c-49ce-4404-8afd-b3513a4568c2 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:59.862198 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2482ac2c-49ce-4404-8afd-b3513a4568c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:56:59.895662 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-2482ac2c-49ce-4404-8afd-b3513a4568c2 None None] MFA Rules not processed for user `8237f2e820ba4eaa9f730afac10288b9`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:56:59.896806 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-2482ac2c-49ce-4404-8afd-b3513a4568c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:56:59.905851 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-81d3b1cf-430a-4a79-a587-5af3cf8198ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0618aad2b69470c9f8607d2f5925eea'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:59.915730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-81d3b1cf-430a-4a79-a587-5af3cf8198ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0618aad2b69470c9f8607d2f5925eea'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:59.975000 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-81d3b1cf-430a-4a79-a587-5af3cf8198ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0618aad2b69470c9f8607d2f5925eea'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:56:59.989245 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1121/2242] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:56:59 2022] DELETE /identity/v3/users/e0618aad2b69470c9f8607d2f5925eea => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 10:56:59.994614 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ddea7-192a-47ca-91e8-a20eba548e4a None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:56:59.994903 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ddea7-192a-47ca-91e8-a20eba548e4a None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:56:59.995229 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ddea7-192a-47ca-91e8-a20eba548e4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:00.016269 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-761ddea7-192a-47ca-91e8-a20eba548e4a None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:57:00.017496 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-761ddea7-192a-47ca-91e8-a20eba548e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:00.029697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1122/2243] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:59 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:57:00.042200 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482ac2c-49ce-4404-8afd-b3513a4568c2 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:57:00.045584 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-2482ac2c-49ce-4404-8afd-b3513a4568c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:00.093700 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1122/2244] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:56:59 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:57:00.105062 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ddea7-192a-47ca-91e8-a20eba548e4a None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:57:00.106017 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:57:00.107215 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-761ddea7-192a-47ca-91e8-a20eba548e4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:00.111785 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2482ac2c-49ce-4404-8afd-b3513a4568c2 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:57:00.166347 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:57:00.172947 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b3b728-ec2a-427b-bf35-a807a78b0b4f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:57:00.173676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3b728-ec2a-427b-bf35-a807a78b0b4f service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:00.174023 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3b728-ec2a-427b-bf35-a807a78b0b4f service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:00.174366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b3b728-ec2a-427b-bf35-a807a78b0b4f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:00.182955 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ddea7-192a-47ca-91e8-a20eba548e4a None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:57:00.189771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6263b18-abde-4d50-8081-823b8392219c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:57:00.190440 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6263b18-abde-4d50-8081-823b8392219c service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:00.190722 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6263b18-abde-4d50-8081-823b8392219c service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:00.191002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6263b18-abde-4d50-8081-823b8392219c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:00.201317 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6263b18-abde-4d50-8081-823b8392219c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:57:00.202895 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6263b18-abde-4d50-8081-823b8392219c service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:57:00.214500 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1123/2245] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:57:00 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:57:00.249991 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b3b728-ec2a-427b-bf35-a807a78b0b4f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:57:00.251697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b3b728-ec2a-427b-bf35-a807a78b0b4f service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:57:00.263325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1123/2246] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:57:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:57:00.774883 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6263b18-abde-4d50-8081-823b8392219c service neutron] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:57:00.776403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-d6263b18-abde-4d50-8081-823b8392219c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:00.802535 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6263b18-abde-4d50-8081-823b8392219c service neutron] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:57:00.808200 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e851f-dc66-44dc-8fa5-1edf1f3034e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:57:00.809557 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e851f-dc66-44dc-8fa5-1edf1f3034e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:00.809827 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e851f-dc66-44dc-8fa5-1edf1f3034e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:00.810181 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e851f-dc66-44dc-8fa5-1edf1f3034e8 admin admin] PATH_INFO: `/v3/projects/6d6b474d83d54803b15bdfd895f0b960` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:00.811574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e851f-dc66-44dc-8fa5-1edf1f3034e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d6b474d83d54803b15bdfd895f0b960)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:57:00.814315 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538e851f-dc66-44dc-8fa5-1edf1f3034e8 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:57:00.968226 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-538e851f-dc66-44dc-8fa5-1edf1f3034e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d6b474d83d54803b15bdfd895f0b960', 'request_id': 'req-538e851f-dc66-44dc-8fa5-1edf1f3034e8'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:57:00.975935 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-538e851f-dc66-44dc-8fa5-1edf1f3034e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d6b474d83d54803b15bdfd895f0b960', 'request_id': 'req-538e851f-dc66-44dc-8fa5-1edf1f3034e8'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:57:01.079570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-538e851f-dc66-44dc-8fa5-1edf1f3034e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d6b474d83d54803b15bdfd895f0b960', 'request_id': 'req-538e851f-dc66-44dc-8fa5-1edf1f3034e8'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:57:01.096178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-538e851f-dc66-44dc-8fa5-1edf1f3034e8 admin admin] The token cache is being invalidate because project 6d6b474d83d54803b15bdfd895f0b960 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:57:01.096560 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-538e851f-dc66-44dc-8fa5-1edf1f3034e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:57:01.097864 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-538e851f-dc66-44dc-8fa5-1edf1f3034e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:57:01.099543 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1124/2247] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:57:00 2022] DELETE /identity/v3/projects/6d6b474d83d54803b15bdfd895f0b960 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:57:01.105274 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b2938-bca9-48f2-ac30-4ce72e03ef94 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:01.105274 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b2938-bca9-48f2-ac30-4ce72e03ef94 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:01.105655 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b2938-bca9-48f2-ac30-4ce72e03ef94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:01.132841 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-0f8b2938-bca9-48f2-ac30-4ce72e03ef94 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:57:01.134055 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-0f8b2938-bca9-48f2-ac30-4ce72e03ef94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:01.227165 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1124/2248] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:57:01 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 10:57:01.231791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cae1d-0bb7-4ab9-875d-8dde3f67d2a6 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:01.232296 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cae1d-0bb7-4ab9-875d-8dde3f67d2a6 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:01.233783 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626cae1d-0bb7-4ab9-875d-8dde3f67d2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:01.257513 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-626cae1d-0bb7-4ab9-875d-8dde3f67d2a6 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:57:01.258863 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-626cae1d-0bb7-4ab9-875d-8dde3f67d2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:01.267196 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1125/2249] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:57:01 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:57:01.271546 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b2938-bca9-48f2-ac30-4ce72e03ef94 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:57:01.282345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b2938-bca9-48f2-ac30-4ce72e03ef94 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:57:01.287403 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69838b-6556-4152-ab2e-73afd6a67659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:57:01.287999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69838b-6556-4152-ab2e-73afd6a67659 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:01.288282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69838b-6556-4152-ab2e-73afd6a67659 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:01.288660 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a69838b-6556-4152-ab2e-73afd6a67659 admin admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:01.289369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a69838b-6556-4152-ab2e-73afd6a67659 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 10:57:01.289677 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a69838b-6556-4152-ab2e-73afd6a67659 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:57:01.291471 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a69838b-6556-4152-ab2e-73afd6a67659 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:57:01.305258 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1125/2250] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 10:57:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 10:57:01.309788 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626cae1d-0bb7-4ab9-875d-8dde3f67d2a6 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:57:01.319331 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626cae1d-0bb7-4ab9-875d-8dde3f67d2a6 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:57:01.324227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506f1dd-0eb1-4b51-a3e3-76ff6cefd10a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:57:01.325155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506f1dd-0eb1-4b51-a3e3-76ff6cefd10a admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:01.325155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506f1dd-0eb1-4b51-a3e3-76ff6cefd10a admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:01.325444 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506f1dd-0eb1-4b51-a3e3-76ff6cefd10a admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:01.326768 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2506f1dd-0eb1-4b51-a3e3-76ff6cefd10a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:57:01.328221 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2506f1dd-0eb1-4b51-a3e3-76ff6cefd10a admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:57:01.435529 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1126/2251] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:57:01 2022] POST /identity/v3/projects => generated 374 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:57:01.439790 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69838b-6556-4152-ab2e-73afd6a67659 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:57:01.450559 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a69838b-6556-4152-ab2e-73afd6a67659 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:57:01.499374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:57:01.515489 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c0387-bdb4-4d86-aa73-87cc75738475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:57:01.516105 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c0387-bdb4-4d86-aa73-87cc75738475 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:01.516379 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c0387-bdb4-4d86-aa73-87cc75738475 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:01.516650 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6c0387-bdb4-4d86-aa73-87cc75738475 admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:01.517319 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6c0387-bdb4-4d86-aa73-87cc75738475 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:57:01.519881 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6c0387-bdb4-4d86-aa73-87cc75738475 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:57:01.669105 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1126/2252] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:57:01 2022] POST /identity/v3/users => generated 288 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:57:01.673096 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506f1dd-0eb1-4b51-a3e3-76ff6cefd10a admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:57:01.685551 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506f1dd-0eb1-4b51-a3e3-76ff6cefd10a admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:57:01.696728 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b0e4f-d2aa-4c98-97b1-0af8a919d043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:57:01.697110 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b0e4f-d2aa-4c98-97b1-0af8a919d043 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:01.697394 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b0e4f-d2aa-4c98-97b1-0af8a919d043 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:01.697655 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8b0e4f-d2aa-4c98-97b1-0af8a919d043 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:01.698407 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b0e4f-d2aa-4c98-97b1-0af8a919d043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:57:01.701056 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8b0e4f-d2aa-4c98-97b1-0af8a919d043 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:57:01.716832 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1127/2253] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:57:01 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:57:01.719007 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c0387-bdb4-4d86-aa73-87cc75738475 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:57:01.731090 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6c0387-bdb4-4d86-aa73-87cc75738475 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:57:01.737882 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881e234e-2f70-4020-aed2-d6bb8a5efda8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:57:01.738444 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e234e-2f70-4020-aed2-d6bb8a5efda8 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:01.739093 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e234e-2f70-4020-aed2-d6bb8a5efda8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:01.739093 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881e234e-2f70-4020-aed2-d6bb8a5efda8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:01.739842 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881e234e-2f70-4020-aed2-d6bb8a5efda8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:57:01.742070 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-881e234e-2f70-4020-aed2-d6bb8a5efda8 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:57:01.759107 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1127/2254] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:57:01 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:57:01.761220 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b0e4f-d2aa-4c98-97b1-0af8a919d043 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:57:01.778162 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8b0e4f-d2aa-4c98-97b1-0af8a919d043 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:57:01.785835 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f09fbe-c54d-4bf9-8e16-f66f882ea8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:57:01.786676 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f09fbe-c54d-4bf9-8e16-f66f882ea8de admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:01.786676 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f09fbe-c54d-4bf9-8e16-f66f882ea8de admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:01.787002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f09fbe-c54d-4bf9-8e16-f66f882ea8de admin admin] PATH_INFO: `/v3/projects/cd54719e5f2b41e6a875361a9188d49b/users/2664b70cbc1a41a59f78b0b7d4e86643/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:01.807201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f09fbe-c54d-4bf9-8e16-f66f882ea8de admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd54719e5f2b41e6a875361a9188d49b, user_id=2664b70cbc1a41a59f78b0b7d4e86643, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:57:01.810364 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f09fbe-c54d-4bf9-8e16-f66f882ea8de admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:57:01.930121 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1128/2255] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:57:01 2022] PUT /identity/v3/projects/cd54719e5f2b41e6a875361a9188d49b/users/2664b70cbc1a41a59f78b0b7d4e86643/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:57:01.931779 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fdb5ff-b4ba-40a8-8483-4376482f78be None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:01.932067 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fdb5ff-b4ba-40a8-8483-4376482f78be None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:01.932361 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fdb5ff-b4ba-40a8-8483-4376482f78be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:01.957924 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-a3fdb5ff-b4ba-40a8-8483-4376482f78be None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:57:01.962509 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-a3fdb5ff-b4ba-40a8-8483-4376482f78be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:02.026645 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1128/2256] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:57:01 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:57:02.037564 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f09fbe-c54d-4bf9-8e16-f66f882ea8de admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:57:02.039436 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-52f09fbe-c54d-4bf9-8e16-f66f882ea8de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:02.108923 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f09fbe-c54d-4bf9-8e16-f66f882ea8de admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:57:02.112731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a332fae4-7cd1-42e0-a539-471a1eb14d36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:57:02.113363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332fae4-7cd1-42e0-a539-471a1eb14d36 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:02.113649 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332fae4-7cd1-42e0-a539-471a1eb14d36 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:02.113922 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a332fae4-7cd1-42e0-a539-471a1eb14d36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:02.123690 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a332fae4-7cd1-42e0-a539-471a1eb14d36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:57:02.125217 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a332fae4-7cd1-42e0-a539-471a1eb14d36 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:57:02.136046 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1129/2257] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:57:02 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:57:04.630096 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630d0b4-17d0-44c4-a6a3-53fa31d8f541 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:04.631414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630d0b4-17d0-44c4-a6a3-53fa31d8f541 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:04.631699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8630d0b4-17d0-44c4-a6a3-53fa31d8f541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:04.670370 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-8630d0b4-17d0-44c4-a6a3-53fa31d8f541 None None] MFA Rules not processed for user `2b03263937cb4ca7807a24da44d37b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:57:04.671730 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-8630d0b4-17d0-44c4-a6a3-53fa31d8f541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:04.803755 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1129/2258] 10.222.0.115 () {58 vars in 1014 bytes} [Tue Aug 2 10:57:04 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:57:04.903779 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a332fae4-7cd1-42e0-a539-471a1eb14d36 service neutron] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:57:04.906823 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-a332fae4-7cd1-42e0-a539-471a1eb14d36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:04.924078 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:57:05.092396 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a332fae4-7cd1-42e0-a539-471a1eb14d36 service neutron] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:57:05.101206 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4e721-f063-42c5-b207-eaf72b77aac4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0daa49b5a35f4a468bb8d0939e7ec20c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:57:05.102176 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4e721-f063-42c5-b207-eaf72b77aac4 service swift] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:05.102702 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4e721-f063-42c5-b207-eaf72b77aac4 service swift] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:05.103307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c4e721-f063-42c5-b207-eaf72b77aac4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:05.117052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4e721-f063-42c5-b207-eaf72b77aac4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:57:05.119704 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c4e721-f063-42c5-b207-eaf72b77aac4 service swift] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:57:05.139514 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1130/2259] 10.222.0.115 () {60 vars in 1389 bytes} [Tue Aug 2 10:57:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 02 10:57:11.394593 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f6916-dbb2-4d66-a990-374ede6f14e1 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:11.395234 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f6916-dbb2-4d66-a990-374ede6f14e1 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:11.395418 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f6916-dbb2-4d66-a990-374ede6f14e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:11.434935 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-c76f6916-dbb2-4d66-a990-374ede6f14e1 None None] MFA Rules not processed for user `2664b70cbc1a41a59f78b0b7d4e86643`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:57:11.437012 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-c76f6916-dbb2-4d66-a990-374ede6f14e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:11.541041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:57:11.549929 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1130/2260] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:57:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:57:11.554363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4e721-f063-42c5-b207-eaf72b77aac4 service swift] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:57:11.568358 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c4e721-f063-42c5-b207-eaf72b77aac4 service swift] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:57:11.575037 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be0907-5a63-4e89-a837-ce76628bf0a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:57:11.575821 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be0907-5a63-4e89-a837-ce76628bf0a1 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:11.576183 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be0907-5a63-4e89-a837-ce76628bf0a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:11.576527 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be0907-5a63-4e89-a837-ce76628bf0a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:11.577381 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be0907-5a63-4e89-a837-ce76628bf0a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:57:11.579485 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be0907-5a63-4e89-a837-ce76628bf0a1 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:57:11.747752 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1131/2261] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 10:57:11 2022] POST /identity/v3/projects => generated 372 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 10:57:11.769863 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f6916-dbb2-4d66-a990-374ede6f14e1 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:57:11.793942 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f6916-dbb2-4d66-a990-374ede6f14e1 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:57:11.920885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ecbfb-619c-4e21-9fef-1cc102d94e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:57:11.921902 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ecbfb-619c-4e21-9fef-1cc102d94e21 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:11.923324 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ecbfb-619c-4e21-9fef-1cc102d94e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:11.923777 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52ecbfb-619c-4e21-9fef-1cc102d94e21 admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:11.924820 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ecbfb-619c-4e21-9fef-1cc102d94e21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:57:11.926751 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52ecbfb-619c-4e21-9fef-1cc102d94e21 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:57:12.071178 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1131/2262] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 10:57:11 2022] POST /identity/v3/users => generated 293 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 10:57:12.075835 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be0907-5a63-4e89-a837-ce76628bf0a1 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:57:12.090701 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be0907-5a63-4e89-a837-ce76628bf0a1 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:57:12.096430 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be24cfea-56f3-4fbf-ab85-d69bc6f1fa44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:57:12.099384 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24cfea-56f3-4fbf-ab85-d69bc6f1fa44 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:12.099384 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24cfea-56f3-4fbf-ab85-d69bc6f1fa44 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:12.099384 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24cfea-56f3-4fbf-ab85-d69bc6f1fa44 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:12.099384 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be24cfea-56f3-4fbf-ab85-d69bc6f1fa44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:57:12.100064 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be24cfea-56f3-4fbf-ab85-d69bc6f1fa44 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:57:12.113767 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1132/2263] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:57:12 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 10:57:12.118665 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ecbfb-619c-4e21-9fef-1cc102d94e21 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:57:12.131995 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52ecbfb-619c-4e21-9fef-1cc102d94e21 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:57:12.141957 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc0025-2818-491e-b22b-d00e79300c41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:57:12.142679 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc0025-2818-491e-b22b-d00e79300c41 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:12.142962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc0025-2818-491e-b22b-d00e79300c41 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:12.143290 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cc0025-2818-491e-b22b-d00e79300c41 admin admin] PATH_INFO: `/v3/projects/4cf6a5aa45e14f8d96889a057e0c6cfe/users/8452abedb4a74e4cad5e804ccd9b8a03/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:12.165422 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc0025-2818-491e-b22b-d00e79300c41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cf6a5aa45e14f8d96889a057e0c6cfe, user_id=8452abedb4a74e4cad5e804ccd9b8a03, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:57:12.169551 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cc0025-2818-491e-b22b-d00e79300c41 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:57:12.267146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1132/2264] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:57:12 2022] PUT /identity/v3/projects/4cf6a5aa45e14f8d96889a057e0c6cfe/users/8452abedb4a74e4cad5e804ccd9b8a03/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:57:12.268520 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be24cfea-56f3-4fbf-ab85-d69bc6f1fa44 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:57:12.280126 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be24cfea-56f3-4fbf-ab85-d69bc6f1fa44 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:57:12.382048 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985d0de7-7da1-4e95-aa40-f59917fb9de8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:57:12.382048 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d0de7-7da1-4e95-aa40-f59917fb9de8 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:12.382048 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d0de7-7da1-4e95-aa40-f59917fb9de8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:12.382403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985d0de7-7da1-4e95-aa40-f59917fb9de8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:12.382991 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d0de7-7da1-4e95-aa40-f59917fb9de8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:57:12.386456 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985d0de7-7da1-4e95-aa40-f59917fb9de8 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:57:12.407223 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1133/2265] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 10:57:12 2022] GET /identity/v3/roles => generated 1427 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 10:57:12.408811 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc0025-2818-491e-b22b-d00e79300c41 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:57:12.427175 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cc0025-2818-491e-b22b-d00e79300c41 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:57:12.442794 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a3f8b6-0e37-4f96-965b-386a14302293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:57:12.444261 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3f8b6-0e37-4f96-965b-386a14302293 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:12.444654 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3f8b6-0e37-4f96-965b-386a14302293 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:12.445497 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a3f8b6-0e37-4f96-965b-386a14302293 admin admin] PATH_INFO: `/v3/projects/4cf6a5aa45e14f8d96889a057e0c6cfe/users/8452abedb4a74e4cad5e804ccd9b8a03/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:12.450049 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a3f8b6-0e37-4f96-965b-386a14302293 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cf6a5aa45e14f8d96889a057e0c6cfe, user_id=8452abedb4a74e4cad5e804ccd9b8a03, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:57:12.457341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a3f8b6-0e37-4f96-965b-386a14302293 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:57:12.485891 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1133/2266] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 10:57:12 2022] PUT /identity/v3/projects/4cf6a5aa45e14f8d96889a057e0c6cfe/users/8452abedb4a74e4cad5e804ccd9b8a03/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:57:21.240767 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae9388-9e7e-4875-afeb-ad64969b8601 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:21.240767 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae9388-9e7e-4875-afeb-ad64969b8601 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:21.240767 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cae9388-9e7e-4875-afeb-ad64969b8601 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:21.294715 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-6cae9388-9e7e-4875-afeb-ad64969b8601 None None] MFA Rules not processed for user `8452abedb4a74e4cad5e804ccd9b8a03`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:57:21.296148 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-6cae9388-9e7e-4875-afeb-ad64969b8601 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:21.412469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:57:21.496277 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1134/2267] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:57:21 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:57:21.512313 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a3f8b6-0e37-4f96-965b-386a14302293 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:57:21.518742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-92a3f8b6-0e37-4f96-965b-386a14302293 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:21.592269 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13b918-9559-400e-82cc-9e914070c63b None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:21.592808 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13b918-9559-400e-82cc-9e914070c63b None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:21.596396 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be13b918-9559-400e-82cc-9e914070c63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:21.616844 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-be13b918-9559-400e-82cc-9e914070c63b None None] MFA Rules not processed for user `914de37664914b4cb91e0ce2c3a0e1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:57:21.618950 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-be13b918-9559-400e-82cc-9e914070c63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:21.645426 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a3f8b6-0e37-4f96-965b-386a14302293 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:57:21.654515 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97b30ec-6ab2-426f-82d1-5043b674c35d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:57:21.655806 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b30ec-6ab2-426f-82d1-5043b674c35d service nova] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:21.656220 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b30ec-6ab2-426f-82d1-5043b674c35d service nova] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:21.656566 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97b30ec-6ab2-426f-82d1-5043b674c35d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:21.739272 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1135/2268] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:57:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:57:21.777649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:57:21.798634 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b30ec-6ab2-426f-82d1-5043b674c35d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:57:21.806680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97b30ec-6ab2-426f-82d1-5043b674c35d service nova] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:57:21.820128 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1134/2269] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:57:21 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:57:26.532205 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c64f4-603d-4821-ac46-a8aa2a15cef3 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:26.532839 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c64f4-603d-4821-ac46-a8aa2a15cef3 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:26.532839 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c64f4-603d-4821-ac46-a8aa2a15cef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:26.555801 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-862c64f4-603d-4821-ac46-a8aa2a15cef3 None None] MFA Rules not processed for user `2664b70cbc1a41a59f78b0b7d4e86643`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:57:26.557469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-862c64f4-603d-4821-ac46-a8aa2a15cef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:26.580370 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1136/2270] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:57:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:57:31.969317 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=6 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:57:33.047620 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956858ad-a1d1-4c54-b944-e6e5f6bae516 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:33.052310 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956858ad-a1d1-4c54-b944-e6e5f6bae516 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:33.052310 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956858ad-a1d1-4c54-b944-e6e5f6bae516 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:33.052310 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1135/2271] 10.222.0.115 () {54 vars in 926 bytes} [Tue Aug 2 10:57:33 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:57:33.055542 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6010a9-56c8-4ea9-8978-651889090dd2 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:33.055988 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6010a9-56c8-4ea9-8978-651889090dd2 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:33.056074 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6010a9-56c8-4ea9-8978-651889090dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:33.079985 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-7b6010a9-56c8-4ea9-8978-651889090dd2 None None] MFA Rules not processed for user `d0478bd540f446ec810e2091c46ac49c`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:57:33.081206 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-7b6010a9-56c8-4ea9-8978-651889090dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:33.090764 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1137/2272] 10.222.0.115 () {58 vars in 1023 bytes} [Tue Aug 2 10:57:33 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:57:39.964591 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956858ad-a1d1-4c54-b944-e6e5f6bae516 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:57:39.967217 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-956858ad-a1d1-4c54-b944-e6e5f6bae516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:40.044254 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956858ad-a1d1-4c54-b944-e6e5f6bae516 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:57:40.052481 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183b2ed1-4ef5-4ed3-890c-a5e29159dc47 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e358fd7c1b3c43d7ac89fb3562678266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:57:40.053428 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b2ed1-4ef5-4ed3-890c-a5e29159dc47 service glance] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:40.053697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b2ed1-4ef5-4ed3-890c-a5e29159dc47 service glance] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:40.054174 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183b2ed1-4ef5-4ed3-890c-a5e29159dc47 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:40.066833 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183b2ed1-4ef5-4ed3-890c-a5e29159dc47 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:57:40.068376 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183b2ed1-4ef5-4ed3-890c-a5e29159dc47 service glance] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:57:40.081533 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1136/2273] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:57:39 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:57:40.313613 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c187f-ba5e-45d3-a5d7-f53e434f68f7 None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:40.313613 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c187f-ba5e-45d3-a5d7-f53e434f68f7 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:40.313613 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4c187f-ba5e-45d3-a5d7-f53e434f68f7 None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:40.313613 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1138/2274] 10.222.0.115 () {54 vars in 926 bytes} [Tue Aug 2 10:57:40 2022] GET /identity => generated 271 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:57:40.321020 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2ab88-4d64-4f84-9370-76cf090c9ef6 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:40.321302 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2ab88-4d64-4f84-9370-76cf090c9ef6 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:40.321578 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2ab88-4d64-4f84-9370-76cf090c9ef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:40.342508 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-22c2ab88-4d64-4f84-9370-76cf090c9ef6 None None] MFA Rules not processed for user `98dcb74152ec4375a586ea526e7f07c8`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:57:40.343778 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-22c2ab88-4d64-4f84-9370-76cf090c9ef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:40.395859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1137/2275] 10.222.0.115 () {58 vars in 1023 bytes} [Tue Aug 2 10:57:40 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:57:40.409176 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c187f-ba5e-45d3-a5d7-f53e434f68f7 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:57:40.410978 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-db4c187f-ba5e-45d3-a5d7-f53e434f68f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:40.425411 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4c187f-ba5e-45d3-a5d7-f53e434f68f7 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:57:40.429009 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd16aed-12cc-428f-9d59-83de6db1cb36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:57:40.429687 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd16aed-12cc-428f-9d59-83de6db1cb36 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:40.430030 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd16aed-12cc-428f-9d59-83de6db1cb36 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:40.430366 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd16aed-12cc-428f-9d59-83de6db1cb36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:40.444977 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd16aed-12cc-428f-9d59-83de6db1cb36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:57:40.446429 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd16aed-12cc-428f-9d59-83de6db1cb36 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:57:40.462005 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1139/2276] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:57:40 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:57:40.560102 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307e641-bcc3-4bd0-8dfd-a0e6af4b2647 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:40.560409 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307e641-bcc3-4bd0-8dfd-a0e6af4b2647 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:40.560678 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d307e641-bcc3-4bd0-8dfd-a0e6af4b2647 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:40.562219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1138/2277] 10.222.0.115 () {54 vars in 926 bytes} [Tue Aug 2 10:57:40 2022] GET /identity => generated 271 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 10:57:40.570069 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208bad9-ae15-4098-a463-5c1528fb4f9b None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:40.570363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208bad9-ae15-4098-a463-5c1528fb4f9b None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:40.570684 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c208bad9-ae15-4098-a463-5c1528fb4f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:40.594158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-c208bad9-ae15-4098-a463-5c1528fb4f9b None None] MFA Rules not processed for user `d0478bd540f446ec810e2091c46ac49c`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:57:40.595719 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-c208bad9-ae15-4098-a463-5c1528fb4f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:40.606389 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1140/2278] 10.222.0.115 () {58 vars in 1023 bytes} [Tue Aug 2 10:57:40 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:57:40.617067 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307e641-bcc3-4bd0-8dfd-a0e6af4b2647 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:57:40.629214 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d307e641-bcc3-4bd0-8dfd-a0e6af4b2647 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:57:40.633485 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c333e4d4-afa0-4066-894f-a8afc33c138e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:57:40.634116 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c333e4d4-afa0-4066-894f-a8afc33c138e service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:40.634398 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c333e4d4-afa0-4066-894f-a8afc33c138e service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:40.634672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c333e4d4-afa0-4066-894f-a8afc33c138e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:40.651210 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c333e4d4-afa0-4066-894f-a8afc33c138e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:57:40.651210 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c333e4d4-afa0-4066-894f-a8afc33c138e service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:57:40.663010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1139/2279] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:57:40 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:57:48.273138 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a85ee-5789-4c6d-9b54-0500fafa93f4 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:48.273693 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a85ee-5789-4c6d-9b54-0500fafa93f4 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:48.273992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a85ee-5789-4c6d-9b54-0500fafa93f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:48.296013 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-fc8a85ee-5789-4c6d-9b54-0500fafa93f4 None None] MFA Rules not processed for user `2b03263937cb4ca7807a24da44d37b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:57:48.297344 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-fc8a85ee-5789-4c6d-9b54-0500fafa93f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:48.361904 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1141/2280] 10.222.0.115 () {58 vars in 1014 bytes} [Tue Aug 2 10:57:48 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:57:48.374197 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c333e4d4-afa0-4066-894f-a8afc33c138e service neutron] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:57:48.452540 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c333e4d4-afa0-4066-894f-a8afc33c138e service neutron] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:57:48.457736 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e9a512-abb9-4338-813f-805542ca32c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0daa49b5a35f4a468bb8d0939e7ec20c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:57:48.459354 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9a512-abb9-4338-813f-805542ca32c5 service swift] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:48.459354 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9a512-abb9-4338-813f-805542ca32c5 service swift] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:48.459354 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e9a512-abb9-4338-813f-805542ca32c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:48.471514 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e9a512-abb9-4338-813f-805542ca32c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:57:48.473130 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e9a512-abb9-4338-813f-805542ca32c5 service swift] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:57:48.483300 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1140/2281] 10.222.0.115 () {60 vars in 1389 bytes} [Tue Aug 2 10:57:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:57:48.917011 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a85ee-5789-4c6d-9b54-0500fafa93f4 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:57:48.931653 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8a85ee-5789-4c6d-9b54-0500fafa93f4 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:57:48.936218 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cfe34f-b6a7-4ae1-9816-c2a1e7363157 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:57:48.936851 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cfe34f-b6a7-4ae1-9816-c2a1e7363157 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:48.937207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cfe34f-b6a7-4ae1-9816-c2a1e7363157 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:48.937410 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cfe34f-b6a7-4ae1-9816-c2a1e7363157 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:48.939316 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-34cfe34f-b6a7-4ae1-9816-c2a1e7363157 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:49.052715 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:57:49.058365 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cfe34f-b6a7-4ae1-9816-c2a1e7363157 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:57:49.060017 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cfe34f-b6a7-4ae1-9816-c2a1e7363157 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:57:49.076458 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1142/2282] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:57:48 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:57:54.080130 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e9a512-abb9-4338-813f-805542ca32c5 service swift] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:57:54.081826 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-77e9a512-abb9-4338-813f-805542ca32c5 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:54.118942 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:57:54.171554 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e9a512-abb9-4338-813f-805542ca32c5 service swift] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:57:54.185149 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168d95d-687f-4460-9e02-ebfad80b8491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:57:54.185813 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168d95d-687f-4460-9e02-ebfad80b8491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:54.186080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168d95d-687f-4460-9e02-ebfad80b8491 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:54.186350 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2168d95d-687f-4460-9e02-ebfad80b8491 admin admin] PATH_INFO: `/v3/users/8237f2e820ba4eaa9f730afac10288b9` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:54.187597 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168d95d-687f-4460-9e02-ebfad80b8491 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8237f2e820ba4eaa9f730afac10288b9)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:57:54.189538 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2168d95d-687f-4460-9e02-ebfad80b8491 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:57:54.417667 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2168d95d-687f-4460-9e02-ebfad80b8491 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8237f2e820ba4eaa9f730afac10288b9'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:57:54.425721 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2168d95d-687f-4460-9e02-ebfad80b8491 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8237f2e820ba4eaa9f730afac10288b9'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:57:54.426604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2168d95d-687f-4460-9e02-ebfad80b8491 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8237f2e820ba4eaa9f730afac10288b9'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:57:54.494185 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2168d95d-687f-4460-9e02-ebfad80b8491 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8237f2e820ba4eaa9f730afac10288b9'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:57:54.509828 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2168d95d-687f-4460-9e02-ebfad80b8491 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8237f2e820ba4eaa9f730afac10288b9'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:57:54.517068 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1141/2283] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 10:57:54 2022] DELETE /identity/v3/users/8237f2e820ba4eaa9f730afac10288b9 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:57:54.521719 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a03fd-56c2-4407-a7c4-3835eb6fcbc8 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:54.522283 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a03fd-56c2-4407-a7c4-3835eb6fcbc8 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:54.522481 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9a03fd-56c2-4407-a7c4-3835eb6fcbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:54.543616 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-da9a03fd-56c2-4407-a7c4-3835eb6fcbc8 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:57:54.545241 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-da9a03fd-56c2-4407-a7c4-3835eb6fcbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:54.619228 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1143/2284] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:57:54 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:57:54.623905 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168d95d-687f-4460-9e02-ebfad80b8491 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:57:54.625389 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-2168d95d-687f-4460-9e02-ebfad80b8491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:54.687488 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2168d95d-687f-4460-9e02-ebfad80b8491 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:57:54.691223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27ad42-047e-478b-bc1c-4ae4b8eae752 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:57:54.691854 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27ad42-047e-478b-bc1c-4ae4b8eae752 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:54.692191 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27ad42-047e-478b-bc1c-4ae4b8eae752 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:54.692528 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa27ad42-047e-478b-bc1c-4ae4b8eae752 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:54.702512 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27ad42-047e-478b-bc1c-4ae4b8eae752 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:57:54.704118 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa27ad42-047e-478b-bc1c-4ae4b8eae752 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:57:54.714414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1142/2285] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:57:54 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 10:57:55.238376 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a03fd-56c2-4407-a7c4-3835eb6fcbc8 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:57:55.240628 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-da9a03fd-56c2-4407-a7c4-3835eb6fcbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:55.256432 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9a03fd-56c2-4407-a7c4-3835eb6fcbc8 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:57:55.261644 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1eaadb-82fb-4dd4-acdb-d0300ba5f813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:57:55.263847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1eaadb-82fb-4dd4-acdb-d0300ba5f813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:55.264242 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1eaadb-82fb-4dd4-acdb-d0300ba5f813 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:55.264614 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1eaadb-82fb-4dd4-acdb-d0300ba5f813 admin admin] PATH_INFO: `/v3/projects/eccf25885e8f49d988e99f5ce9ad8d4a` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:55.265877 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1eaadb-82fb-4dd4-acdb-d0300ba5f813 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eccf25885e8f49d988e99f5ce9ad8d4a)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:57:55.267920 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1eaadb-82fb-4dd4-acdb-d0300ba5f813 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:57:55.402643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-bc1eaadb-82fb-4dd4-acdb-d0300ba5f813 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eccf25885e8f49d988e99f5ce9ad8d4a', 'request_id': 'req-bc1eaadb-82fb-4dd4-acdb-d0300ba5f813'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:57:55.412064 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-bc1eaadb-82fb-4dd4-acdb-d0300ba5f813 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eccf25885e8f49d988e99f5ce9ad8d4a', 'request_id': 'req-bc1eaadb-82fb-4dd4-acdb-d0300ba5f813'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:57:55.483086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-bc1eaadb-82fb-4dd4-acdb-d0300ba5f813 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eccf25885e8f49d988e99f5ce9ad8d4a', 'request_id': 'req-bc1eaadb-82fb-4dd4-acdb-d0300ba5f813'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:57:55.498264 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-bc1eaadb-82fb-4dd4-acdb-d0300ba5f813 admin admin] The token cache is being invalidate because project eccf25885e8f49d988e99f5ce9ad8d4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 10:57:55.498631 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-bc1eaadb-82fb-4dd4-acdb-d0300ba5f813 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:57:55.499402 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-bc1eaadb-82fb-4dd4-acdb-d0300ba5f813 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 10:57:55.500923 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1144/2286] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 10:57:55 2022] DELETE /identity/v3/projects/eccf25885e8f49d988e99f5ce9ad8d4a => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 10:57:59.856832 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27ad42-047e-478b-bc1c-4ae4b8eae752 service neutron] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:57:59.858618 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-fa27ad42-047e-478b-bc1c-4ae4b8eae752 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:59.880352 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1eaadb-82fb-4dd4-acdb-d0300ba5f813 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:57:59.881780 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-bc1eaadb-82fb-4dd4-acdb-d0300ba5f813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:59.927673 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:57:59.946537 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa27ad42-047e-478b-bc1c-4ae4b8eae752 service neutron] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:57:59.963846 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1eaadb-82fb-4dd4-acdb-d0300ba5f813 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:57:59.969654 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa7af8-a159-4f22-86bd-d4a165799894 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc4b962e2a742b08beef7187cc527e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:57:59.970310 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa7af8-a159-4f22-86bd-d4a165799894 service placement] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:57:59.970591 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa7af8-a159-4f22-86bd-d4a165799894 service placement] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:57:59.970868 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa7af8-a159-4f22-86bd-d4a165799894 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:57:59.972681 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-71aa7af8-a159-4f22-86bd-d4a165799894 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:57:59.985714 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa7af8-a159-4f22-86bd-d4a165799894 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:57:59.987370 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa7af8-a159-4f22-86bd-d4a165799894 service placement] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:57:59.998886 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1145/2287] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:57:59 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:58:00.014871 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac29c6f-665c-4b35-a53c-9a0ed8db66d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc4b962e2a742b08beef7187cc527e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:58:00.015805 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac29c6f-665c-4b35-a53c-9a0ed8db66d2 service placement] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:58:00.016087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac29c6f-665c-4b35-a53c-9a0ed8db66d2 service placement] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:58:00.016364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac29c6f-665c-4b35-a53c-9a0ed8db66d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:58:00.026164 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac29c6f-665c-4b35-a53c-9a0ed8db66d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:58:00.027855 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac29c6f-665c-4b35-a53c-9a0ed8db66d2 service placement] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:58:00.040691 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1143/2288] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:57:59 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:58:08.596683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d486e-e2e1-40d8-ba4d-9f5d1d1e7882 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:58:08.598003 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d486e-e2e1-40d8-ba4d-9f5d1d1e7882 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:58:08.598003 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d486e-e2e1-40d8-ba4d-9f5d1d1e7882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:58:08.619323 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-297d486e-e2e1-40d8-ba4d-9f5d1d1e7882 None None] MFA Rules not processed for user `2b03263937cb4ca7807a24da44d37b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:58:08.620557 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-297d486e-e2e1-40d8-ba4d-9f5d1d1e7882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:58:08.626328 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:58:08.687632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1146/2289] 10.222.0.115 () {58 vars in 972 bytes} [Tue Aug 2 10:58:08 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:58:08.711494 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac29c6f-665c-4b35-a53c-9a0ed8db66d2 service placement] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:58:08.713139 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-1ac29c6f-665c-4b35-a53c-9a0ed8db66d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:58:08.795465 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac29c6f-665c-4b35-a53c-9a0ed8db66d2 service placement] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:58:08.801961 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642644f4-0324-4d75-b5af-40965a27e439 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0daa49b5a35f4a468bb8d0939e7ec20c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:58:08.803296 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642644f4-0324-4d75-b5af-40965a27e439 service swift] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:58:08.804013 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642644f4-0324-4d75-b5af-40965a27e439 service swift] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:58:08.804473 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642644f4-0324-4d75-b5af-40965a27e439 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:58:08.818363 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642644f4-0324-4d75-b5af-40965a27e439 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:58:08.820011 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642644f4-0324-4d75-b5af-40965a27e439 service swift] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:58:08.829064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1144/2290] 10.222.0.115 () {60 vars in 1389 bytes} [Tue Aug 2 10:58:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 10:58:18.699452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=21 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:58:18.825402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=67 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:58:20.985731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d486e-e2e1-40d8-ba4d-9f5d1d1e7882 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:58:20.987339 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-297d486e-e2e1-40d8-ba4d-9f5d1d1e7882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:58:21.049691 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d486e-e2e1-40d8-ba4d-9f5d1d1e7882 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:58:21.053497 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea7ae58-8af6-463d-a0eb-b694f053c50f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:58:21.054148 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7ae58-8af6-463d-a0eb-b694f053c50f service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:58:21.054429 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7ae58-8af6-463d-a0eb-b694f053c50f service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:58:21.054714 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea7ae58-8af6-463d-a0eb-b694f053c50f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:58:21.056557 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-2ea7ae58-8af6-463d-a0eb-b694f053c50f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:58:21.128552 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea7ae58-8af6-463d-a0eb-b694f053c50f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:58:21.130011 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea7ae58-8af6-463d-a0eb-b694f053c50f service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:58:21.141375 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1147/2291] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:58:20 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:58:28.467541 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d738a40-318c-4065-90b7-594ba159ddb7 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:58:28.468037 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d738a40-318c-4065-90b7-594ba159ddb7 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:58:28.468181 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d738a40-318c-4065-90b7-594ba159ddb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:58:28.488533 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-3d738a40-318c-4065-90b7-594ba159ddb7 None None] MFA Rules not processed for user `3057a53debc34c829372c03567e1119d`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:58:28.489772 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-3d738a40-318c-4065-90b7-594ba159ddb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:58:28.548138 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1145/2292] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:58:28 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:58:31.135555 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=50 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:58:34.644281 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f59fd-6d3b-4658-adea-a53166bf9916 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:58:34.644967 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f59fd-6d3b-4658-adea-a53166bf9916 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:58:34.644967 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f59fd-6d3b-4658-adea-a53166bf9916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:58:34.666028 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-b61f59fd-6d3b-4658-adea-a53166bf9916 None None] MFA Rules not processed for user `53bc14ada1564911b2f0a698aef3f333`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:58:34.667284 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-b61f59fd-6d3b-4658-adea-a53166bf9916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:58:34.722910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1148/2293] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:58:34 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:58:38.540606 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=25 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:58:44.716209 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=25 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:59:13.744209 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa2b46-7d3d-4a37-9882-fb8b143e6ccc None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:59:13.744841 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa2b46-7d3d-4a37-9882-fb8b143e6ccc None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:59:13.744841 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa2b46-7d3d-4a37-9882-fb8b143e6ccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:59:13.765366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-f4fa2b46-7d3d-4a37-9882-fb8b143e6ccc None None] MFA Rules not processed for user `ab1211acdd9a4df48c46f8807ed00063`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:59:13.766612 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f4fa2b46-7d3d-4a37-9882-fb8b143e6ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:59:13.822676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1146/2294] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:59:13 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:59:23.817239 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=25 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 10:59:52.809887 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f59fd-6d3b-4658-adea-a53166bf9916 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:59:52.819899 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f59fd-6d3b-4658-adea-a53166bf9916 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:59:52.825427 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4dba66-2ed7-478c-bb09-31993c59522a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:59:52.826244 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4dba66-2ed7-478c-bb09-31993c59522a service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:59:52.826699 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4dba66-2ed7-478c-bb09-31993c59522a service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:59:52.827038 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4dba66-2ed7-478c-bb09-31993c59522a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:59:52.829335 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-0a4dba66-2ed7-478c-bb09-31993c59522a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:59:52.915313 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4dba66-2ed7-478c-bb09-31993c59522a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:59:52.917843 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4dba66-2ed7-478c-bb09-31993c59522a service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:59:52.929251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1149/2295] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:59:52 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 10:59:54.121805 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa47c3-1923-439e-9d53-61273383bf92 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:59:54.122383 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa47c3-1923-439e-9d53-61273383bf92 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:59:54.122383 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa47c3-1923-439e-9d53-61273383bf92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:59:54.141741 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-6faa47c3-1923-439e-9d53-61273383bf92 None None] MFA Rules not processed for user `f3f41f17f5154e01a699043edc350cf8`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 10:59:54.142966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-6faa47c3-1923-439e-9d53-61273383bf92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:59:54.192288 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4dba66-2ed7-478c-bb09-31993c59522a service neutron] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 10:59:54.201836 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4dba66-2ed7-478c-bb09-31993c59522a service neutron] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 10:59:54.202327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1147/2296] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 10:59:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 10:59:54.205738 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24806ad-649d-4f17-a353-adf6e79f5d46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 10:59:54.206392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24806ad-649d-4f17-a353-adf6e79f5d46 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 10:59:54.206673 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24806ad-649d-4f17-a353-adf6e79f5d46 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 10:59:54.206947 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24806ad-649d-4f17-a353-adf6e79f5d46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 10:59:54.208750 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-e24806ad-649d-4f17-a353-adf6e79f5d46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 10:59:54.223960 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24806ad-649d-4f17-a353-adf6e79f5d46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 10:59:54.225393 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24806ad-649d-4f17-a353-adf6e79f5d46 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 10:59:54.237807 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1150/2297] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 10:59:54 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:00:04.194459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=25 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:04.235534 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=40 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:08.686075 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a72750-dd36-4c9b-bf3b-23421872dde2 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:08.686628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a72750-dd36-4c9b-bf3b-23421872dde2 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:08.686737 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a72750-dd36-4c9b-bf3b-23421872dde2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:08.706536 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-01a72750-dd36-4c9b-bf3b-23421872dde2 None None] MFA Rules not processed for user `53bc14ada1564911b2f0a698aef3f333`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:00:08.707977 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-01a72750-dd36-4c9b-bf3b-23421872dde2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:08.718171 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1148/2298] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:00:08 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:00:10.551717 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24806ad-649d-4f17-a353-adf6e79f5d46 service neutron] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:10.553248 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-e24806ad-649d-4f17-a353-adf6e79f5d46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:10.569417 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24806ad-649d-4f17-a353-adf6e79f5d46 service neutron] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:10.579784 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb172028-213d-4aba-8a44-a60198682665 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:10.581088 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb172028-213d-4aba-8a44-a60198682665 service nova] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:10.581531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb172028-213d-4aba-8a44-a60198682665 service nova] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:10.581982 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb172028-213d-4aba-8a44-a60198682665 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:10.584241 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-fb172028-213d-4aba-8a44-a60198682665 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:10.597511 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb172028-213d-4aba-8a44-a60198682665 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:10.600413 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb172028-213d-4aba-8a44-a60198682665 service nova] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:10.612361 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1151/2299] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:00:10 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:00:11.281195 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c12c4e-1254-4ea3-9cf9-9f0ca94701ac None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:11.281732 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c12c4e-1254-4ea3-9cf9-9f0ca94701ac None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:11.281824 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c12c4e-1254-4ea3-9cf9-9f0ca94701ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:11.302039 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-83c12c4e-1254-4ea3-9cf9-9f0ca94701ac None None] MFA Rules not processed for user `3057a53debc34c829372c03567e1119d`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:00:11.303267 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-83c12c4e-1254-4ea3-9cf9-9f0ca94701ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:11.311792 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1149/2300] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:00:11 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:00:12.459247 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb172028-213d-4aba-8a44-a60198682665 service nova] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:12.463006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-fb172028-213d-4aba-8a44-a60198682665 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:12.478750 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb172028-213d-4aba-8a44-a60198682665 service nova] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:12.486639 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed11c6-e5f2-4511-a8ee-08df638e02b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:12.487419 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed11c6-e5f2-4511-a8ee-08df638e02b5 service nova] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:12.487773 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed11c6-e5f2-4511-a8ee-08df638e02b5 service nova] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:12.488116 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ed11c6-e5f2-4511-a8ee-08df638e02b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:12.490067 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-c1ed11c6-e5f2-4511-a8ee-08df638e02b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:12.503456 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ed11c6-e5f2-4511-a8ee-08df638e02b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:12.505453 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ed11c6-e5f2-4511-a8ee-08df638e02b5 service nova] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:12.518899 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1152/2301] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:00:12 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:00:17.199968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca4c51a-f13d-4160-a2cd-679ebe0d0cdc None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:17.200537 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca4c51a-f13d-4160-a2cd-679ebe0d0cdc None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:17.200537 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca4c51a-f13d-4160-a2cd-679ebe0d0cdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:17.221285 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-1ca4c51a-f13d-4160-a2cd-679ebe0d0cdc None None] MFA Rules not processed for user `914de37664914b4cb91e0ce2c3a0e1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:00:17.222531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-1ca4c51a-f13d-4160-a2cd-679ebe0d0cdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:17.282194 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1150/2302] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:00:17 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:00:18.652337 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed11c6-e5f2-4511-a8ee-08df638e02b5 service nova] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:18.666600 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ed11c6-e5f2-4511-a8ee-08df638e02b5 service nova] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:18.673043 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a7748-ece2-4054-bad7-5b041ec87e6c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc4b962e2a742b08beef7187cc527e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:18.673977 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a7748-ece2-4054-bad7-5b041ec87e6c service placement] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:18.675353 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a7748-ece2-4054-bad7-5b041ec87e6c service placement] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:18.675805 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5a7748-ece2-4054-bad7-5b041ec87e6c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:18.677642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-2c5a7748-ece2-4054-bad7-5b041ec87e6c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:18.692373 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a7748-ece2-4054-bad7-5b041ec87e6c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:18.693964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5a7748-ece2-4054-bad7-5b041ec87e6c service placement] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:18.737320 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1153/2303] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:00:18 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:00:19.028530 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca4c51a-f13d-4160-a2cd-679ebe0d0cdc None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:19.038134 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca4c51a-f13d-4160-a2cd-679ebe0d0cdc None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:19.042390 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29988b7-266b-416f-9a95-59279995ecf5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:19.043129 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29988b7-266b-416f-9a95-59279995ecf5 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:19.043415 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29988b7-266b-416f-9a95-59279995ecf5 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:19.043711 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29988b7-266b-416f-9a95-59279995ecf5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:19.045456 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f29988b7-266b-416f-9a95-59279995ecf5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:19.056274 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29988b7-266b-416f-9a95-59279995ecf5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:19.057607 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29988b7-266b-416f-9a95-59279995ecf5 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:19.066304 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1151/2304] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:00:19 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:00:28.701556 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=18 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:28.858054 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a7748-ece2-4054-bad7-5b041ec87e6c service placement] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:28.859598 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5a7748-ece2-4054-bad7-5b041ec87e6c service placement] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:28.866488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29a157-13d7-4154-bfbf-0776693fdc5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:28.867223 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29a157-13d7-4154-bfbf-0776693fdc5f service nova] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:28.867574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29a157-13d7-4154-bfbf-0776693fdc5f service nova] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:28.868427 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29a157-13d7-4154-bfbf-0776693fdc5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:28.870285 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-5e29a157-13d7-4154-bfbf-0776693fdc5f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:28.883310 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29a157-13d7-4154-bfbf-0776693fdc5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:28.885978 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29a157-13d7-4154-bfbf-0776693fdc5f service nova] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:28.897474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1154/2305] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:00:28 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:00:29.062283 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=37 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:29.216005 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e82eb0-fb9c-43ca-9903-1daaceded5ad None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:29.216288 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e82eb0-fb9c-43ca-9903-1daaceded5ad None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:29.216552 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e82eb0-fb9c-43ca-9903-1daaceded5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:29.238193 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-17e82eb0-fb9c-43ca-9903-1daaceded5ad None None] MFA Rules not processed for user `2b03263937cb4ca7807a24da44d37b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:00:29.239527 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-17e82eb0-fb9c-43ca-9903-1daaceded5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:29.247791 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1152/2306] 10.222.0.115 () {58 vars in 1014 bytes} [Tue Aug 2 11:00:29 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:00:29.437259 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29a157-13d7-4154-bfbf-0776693fdc5f service nova] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:29.448777 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29a157-13d7-4154-bfbf-0776693fdc5f service nova] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:29.454010 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d20f068-322a-4a54-ae15-1494934c59b3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0daa49b5a35f4a468bb8d0939e7ec20c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:29.454636 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20f068-322a-4a54-ae15-1494934c59b3 service swift] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:29.455265 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20f068-322a-4a54-ae15-1494934c59b3 service swift] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:29.455718 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d20f068-322a-4a54-ae15-1494934c59b3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:29.465201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d20f068-322a-4a54-ae15-1494934c59b3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:29.466867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d20f068-322a-4a54-ae15-1494934c59b3 service swift] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:29.475760 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1155/2307] 10.222.0.115 () {60 vars in 1389 bytes} [Tue Aug 2 11:00:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 11:00:39.233593 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:39.472001 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=12 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:40.034161 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e82eb0-fb9c-43ca-9903-1daaceded5ad None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:40.056299 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e82eb0-fb9c-43ca-9903-1daaceded5ad None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:40.060081 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251f1e12-0c90-42dd-ad2d-cf47760d14b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:40.060879 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f1e12-0c90-42dd-ad2d-cf47760d14b2 service nova] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:40.061336 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f1e12-0c90-42dd-ad2d-cf47760d14b2 service nova] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:40.061763 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251f1e12-0c90-42dd-ad2d-cf47760d14b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:40.063853 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-251f1e12-0c90-42dd-ad2d-cf47760d14b2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:40.080249 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f1e12-0c90-42dd-ad2d-cf47760d14b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:40.082392 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251f1e12-0c90-42dd-ad2d-cf47760d14b2 service nova] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:40.095193 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1153/2308] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:00:40 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:00:44.250187 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d20f068-322a-4a54-ae15-1494934c59b3 service swift] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:44.262452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d20f068-322a-4a54-ae15-1494934c59b3 service swift] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:44.268102 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb91304-ad14-4318-a061-f3ca9a9314c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:44.268746 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb91304-ad14-4318-a061-f3ca9a9314c9 service nova] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:44.269024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb91304-ad14-4318-a061-f3ca9a9314c9 service nova] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:44.269300 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb91304-ad14-4318-a061-f3ca9a9314c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:44.271133 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-fbb91304-ad14-4318-a061-f3ca9a9314c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:44.284867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb91304-ad14-4318-a061-f3ca9a9314c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:44.286374 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb91304-ad14-4318-a061-f3ca9a9314c9 service nova] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:44.297871 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1156/2309] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:00:44 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:00:50.089602 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=6 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:00:50.246415 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d9f38-22ce-4ec9-b216-bade53da34ec None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:50.246696 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d9f38-22ce-4ec9-b216-bade53da34ec None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:50.246966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d9f38-22ce-4ec9-b216-bade53da34ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:50.266233 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-339d9f38-22ce-4ec9-b216-bade53da34ec None None] MFA Rules not processed for user `2b03263937cb4ca7807a24da44d37b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:00:50.267479 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-339d9f38-22ce-4ec9-b216-bade53da34ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:50.274628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1154/2310] 10.222.0.115 () {58 vars in 1014 bytes} [Tue Aug 2 11:00:50 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:00:50.287306 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb91304-ad14-4318-a061-f3ca9a9314c9 service nova] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:50.299633 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb91304-ad14-4318-a061-f3ca9a9314c9 service nova] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:50.304588 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ab815-0f3e-420b-a016-705d96ec45d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0daa49b5a35f4a468bb8d0939e7ec20c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:50.305192 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ab815-0f3e-420b-a016-705d96ec45d1 service swift] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:50.305474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ab815-0f3e-420b-a016-705d96ec45d1 service swift] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:50.305763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ab815-0f3e-420b-a016-705d96ec45d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:50.314242 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ab815-0f3e-420b-a016-705d96ec45d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:50.315852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ab815-0f3e-420b-a016-705d96ec45d1 service swift] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:50.324878 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1157/2311] 10.222.0.115 () {60 vars in 1389 bytes} [Tue Aug 2 11:00:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 11:00:57.878192 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d9f38-22ce-4ec9-b216-bade53da34ec None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:00:57.891889 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d9f38-22ce-4ec9-b216-bade53da34ec None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:00:57.897759 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59601db3-48f8-4c8d-af42-4a4be212f817 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:00:57.900221 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59601db3-48f8-4c8d-af42-4a4be212f817 service nova] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:00:57.900221 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59601db3-48f8-4c8d-af42-4a4be212f817 service nova] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:00:57.900221 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59601db3-48f8-4c8d-af42-4a4be212f817 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:00:57.900714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-59601db3-48f8-4c8d-af42-4a4be212f817 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:00:57.912489 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59601db3-48f8-4c8d-af42-4a4be212f817 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:00:57.914076 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59601db3-48f8-4c8d-af42-4a4be212f817 service nova] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:00:57.925718 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1155/2312] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:00:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:01:00.320702 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=12 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:07.920729 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=9 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:08.415076 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ab815-0f3e-420b-a016-705d96ec45d1 service swift] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:08.426064 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ab815-0f3e-420b-a016-705d96ec45d1 service swift] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:08.432294 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76bb1a-f4bc-4aaf-959e-2abb22f52443 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:08.433260 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76bb1a-f4bc-4aaf-959e-2abb22f52443 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:08.433956 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76bb1a-f4bc-4aaf-959e-2abb22f52443 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:08.434375 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76bb1a-f4bc-4aaf-959e-2abb22f52443 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:08.437120 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-ee76bb1a-f4bc-4aaf-959e-2abb22f52443 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:08.451728 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee76bb1a-f4bc-4aaf-959e-2abb22f52443 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:08.455206 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee76bb1a-f4bc-4aaf-959e-2abb22f52443 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:08.465997 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1158/2313] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:01:08 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:01:08.941829 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016c6a3-bf0f-42a4-a61b-baa241b1f358 None None] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:08.942397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016c6a3-bf0f-42a4-a61b-baa241b1f358 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:08.942751 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016c6a3-bf0f-42a4-a61b-baa241b1f358 None None] PATH_INFO: `/` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:08.945701 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1156/2314] 10.222.0.115 () {54 vars in 926 bytes} [Tue Aug 2 11:01:08 2022] GET /identity => generated 271 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 11:01:08.953536 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681259f6-2402-4f08-86f1-48e2749bcc56 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:08.954043 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681259f6-2402-4f08-86f1-48e2749bcc56 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:08.954287 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681259f6-2402-4f08-86f1-48e2749bcc56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:08.975481 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-681259f6-2402-4f08-86f1-48e2749bcc56 None None] MFA Rules not processed for user `98dcb74152ec4375a586ea526e7f07c8`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:08.976930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-681259f6-2402-4f08-86f1-48e2749bcc56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:08.984702 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1159/2315] 10.222.0.115 () {58 vars in 1023 bytes} [Tue Aug 2 11:01:08 2022] POST /identity/v3/auth/tokens => generated 3131 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:01:09.008774 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016c6a3-bf0f-42a4-a61b-baa241b1f358 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:09.011333 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-3016c6a3-bf0f-42a4-a61b-baa241b1f358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:09.024498 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016c6a3-bf0f-42a4-a61b-baa241b1f358 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:09.029327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79067ed1-978c-46a7-8203-fbae7d7f4944 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:09.029933 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79067ed1-978c-46a7-8203-fbae7d7f4944 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:09.030213 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79067ed1-978c-46a7-8203-fbae7d7f4944 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:09.030487 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79067ed1-978c-46a7-8203-fbae7d7f4944 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:09.038473 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79067ed1-978c-46a7-8203-fbae7d7f4944 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:09.040749 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79067ed1-978c-46a7-8203-fbae7d7f4944 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:09.049952 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1157/2316] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:01:09 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:01:09.133199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b37a41-7a6f-4074-a59e-9a84295a18cc None None] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:09.133521 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b37a41-7a6f-4074-a59e-9a84295a18cc None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:09.133802 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b37a41-7a6f-4074-a59e-9a84295a18cc None None] PATH_INFO: `/` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:09.135952 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1160/2317] 10.222.0.115 () {54 vars in 926 bytes} [Tue Aug 2 11:01:09 2022] GET /identity => generated 271 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 235 bytes (1 switches on core 0) Aug 02 11:01:09.141748 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e8d4c-b1f6-4416-ba89-079e6a8b4b25 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:09.142036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e8d4c-b1f6-4416-ba89-079e6a8b4b25 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:09.142319 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535e8d4c-b1f6-4416-ba89-079e6a8b4b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:09.162087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-535e8d4c-b1f6-4416-ba89-079e6a8b4b25 None None] MFA Rules not processed for user `d0478bd540f446ec810e2091c46ac49c`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:09.163325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-535e8d4c-b1f6-4416-ba89-079e6a8b4b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:09.173138 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1158/2318] 10.222.0.115 () {58 vars in 1023 bytes} [Tue Aug 2 11:01:09 2022] POST /identity/v3/auth/tokens => generated 3382 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:09.184382 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b37a41-7a6f-4074-a59e-9a84295a18cc None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:09.195237 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b37a41-7a6f-4074-a59e-9a84295a18cc None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:09.200199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c555d6-354c-415e-ab38-e6ec3cafb46a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:09.200821 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c555d6-354c-415e-ab38-e6ec3cafb46a service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:09.201095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c555d6-354c-415e-ab38-e6ec3cafb46a service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:09.201368 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c555d6-354c-415e-ab38-e6ec3cafb46a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:09.213095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c555d6-354c-415e-ab38-e6ec3cafb46a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:09.214727 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c555d6-354c-415e-ab38-e6ec3cafb46a service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:09.226263 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1161/2319] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:01:09 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:01:10.782514 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b8e61-93e0-4a22-b791-4a9e401e36aa None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:10.783000 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b8e61-93e0-4a22-b791-4a9e401e36aa None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:10.783152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78b8e61-93e0-4a22-b791-4a9e401e36aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:10.801831 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-f78b8e61-93e0-4a22-b791-4a9e401e36aa None None] MFA Rules not processed for user `2b03263937cb4ca7807a24da44d37b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:10.803031 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f78b8e61-93e0-4a22-b791-4a9e401e36aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:10.811460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1159/2320] 10.222.0.115 () {58 vars in 972 bytes} [Tue Aug 2 11:01:10 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:10.824573 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c555d6-354c-415e-ab38-e6ec3cafb46a service neutron] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:10.835938 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c555d6-354c-415e-ab38-e6ec3cafb46a service neutron] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:10.841291 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526efd8b-cc23-4720-9ef0-9898248ac18e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0daa49b5a35f4a468bb8d0939e7ec20c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:10.841897 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526efd8b-cc23-4720-9ef0-9898248ac18e service swift] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:10.842173 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526efd8b-cc23-4720-9ef0-9898248ac18e service swift] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:10.842445 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526efd8b-cc23-4720-9ef0-9898248ac18e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:10.850900 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526efd8b-cc23-4720-9ef0-9898248ac18e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:10.852488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526efd8b-cc23-4720-9ef0-9898248ac18e service swift] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:10.861252 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1162/2321] 10.222.0.115 () {60 vars in 1389 bytes} [Tue Aug 2 11:01:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 11:01:20.779010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b8e61-93e0-4a22-b791-4a9e401e36aa None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:20.797829 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78b8e61-93e0-4a22-b791-4a9e401e36aa None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:20.803524 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1edffbe-c7fa-43be-b2c6-b12229f7095a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:20.804153 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1edffbe-c7fa-43be-b2c6-b12229f7095a service nova] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:20.804459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1edffbe-c7fa-43be-b2c6-b12229f7095a service nova] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:20.804732 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1edffbe-c7fa-43be-b2c6-b12229f7095a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:20.806657 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-c1edffbe-c7fa-43be-b2c6-b12229f7095a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:20.819141 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1edffbe-c7fa-43be-b2c6-b12229f7095a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:20.820777 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1edffbe-c7fa-43be-b2c6-b12229f7095a service nova] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:20.831356 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1160/2322] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:01:20 2022] GET /identity/v3/auth/tokens => generated 3280 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:01:20.857749 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=21 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:30.826630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=18 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:37.982290 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526efd8b-cc23-4720-9ef0-9898248ac18e service swift] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:37.991584 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526efd8b-cc23-4720-9ef0-9898248ac18e service swift] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:37.995103 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346d83b-f040-4150-9c5f-1216e1fb153a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:37.995709 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346d83b-f040-4150-9c5f-1216e1fb153a service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:37.995988 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346d83b-f040-4150-9c5f-1216e1fb153a service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:37.996300 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2346d83b-f040-4150-9c5f-1216e1fb153a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:37.998286 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-2346d83b-f040-4150-9c5f-1216e1fb153a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:38.061749 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346d83b-f040-4150-9c5f-1216e1fb153a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:38.063276 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2346d83b-f040-4150-9c5f-1216e1fb153a service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:38.072749 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1163/2323] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:01:37 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:01:41.916681 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1edffbe-c7fa-43be-b2c6-b12229f7095a service nova] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:41.927256 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1edffbe-c7fa-43be-b2c6-b12229f7095a service nova] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:41.932263 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33397f91-9295-4188-9942-3eabf59d8ac9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc4b962e2a742b08beef7187cc527e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:41.932858 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33397f91-9295-4188-9942-3eabf59d8ac9 service placement] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:41.933138 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33397f91-9295-4188-9942-3eabf59d8ac9 service placement] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:41.933414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33397f91-9295-4188-9942-3eabf59d8ac9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:41.936438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-33397f91-9295-4188-9942-3eabf59d8ac9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:41.948608 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33397f91-9295-4188-9942-3eabf59d8ac9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:41.951749 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33397f91-9295-4188-9942-3eabf59d8ac9 service placement] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:41.962277 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1161/2324] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:01:41 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:01:42.042411 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346d83b-f040-4150-9c5f-1216e1fb153a service neutron] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:42.044508 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-2346d83b-f040-4150-9c5f-1216e1fb153a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:42.058184 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2346d83b-f040-4150-9c5f-1216e1fb153a service neutron] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:42.062987 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc3e03a-9b49-4ac5-b7bb-cab13f5d3611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:42.063680 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3e03a-9b49-4ac5-b7bb-cab13f5d3611 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:42.063954 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3e03a-9b49-4ac5-b7bb-cab13f5d3611 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:42.064337 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc3e03a-9b49-4ac5-b7bb-cab13f5d3611 admin admin] PATH_INFO: `/v3/users/3057a53debc34c829372c03567e1119d` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:42.065508 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc3e03a-9b49-4ac5-b7bb-cab13f5d3611 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3057a53debc34c829372c03567e1119d)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:42.067683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc3e03a-9b49-4ac5-b7bb-cab13f5d3611 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:42.204447 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-6fc3e03a-9b49-4ac5-b7bb-cab13f5d3611 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3057a53debc34c829372c03567e1119d'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:42.205147 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-6fc3e03a-9b49-4ac5-b7bb-cab13f5d3611 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3057a53debc34c829372c03567e1119d'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:42.252602 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-6fc3e03a-9b49-4ac5-b7bb-cab13f5d3611 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3057a53debc34c829372c03567e1119d'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:42.261722 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-6fc3e03a-9b49-4ac5-b7bb-cab13f5d3611 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3057a53debc34c829372c03567e1119d'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:42.278968 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-6fc3e03a-9b49-4ac5-b7bb-cab13f5d3611 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3057a53debc34c829372c03567e1119d'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:42.285832 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1164/2325] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:01:42 2022] DELETE /identity/v3/users/3057a53debc34c829372c03567e1119d => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:42.553823 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33397f91-9295-4188-9942-3eabf59d8ac9 service placement] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:42.568893 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33397f91-9295-4188-9942-3eabf59d8ac9 service placement] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:42.630259 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a464030-3dee-401d-9082-8bf9ddc8dd52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:42.630893 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a464030-3dee-401d-9082-8bf9ddc8dd52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:42.631200 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a464030-3dee-401d-9082-8bf9ddc8dd52 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:42.631473 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a464030-3dee-401d-9082-8bf9ddc8dd52 admin admin] PATH_INFO: `/v3/projects/0d99d3fd13df429a8ee8a7cb7780adb5` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:42.632626 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a464030-3dee-401d-9082-8bf9ddc8dd52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d99d3fd13df429a8ee8a7cb7780adb5)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:42.634534 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a464030-3dee-401d-9082-8bf9ddc8dd52 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:42.746143 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-6a464030-3dee-401d-9082-8bf9ddc8dd52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d99d3fd13df429a8ee8a7cb7780adb5', 'request_id': 'req-6a464030-3dee-401d-9082-8bf9ddc8dd52'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:42.821400 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-6a464030-3dee-401d-9082-8bf9ddc8dd52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d99d3fd13df429a8ee8a7cb7780adb5', 'request_id': 'req-6a464030-3dee-401d-9082-8bf9ddc8dd52'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:42.829788 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-6a464030-3dee-401d-9082-8bf9ddc8dd52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d99d3fd13df429a8ee8a7cb7780adb5', 'request_id': 'req-6a464030-3dee-401d-9082-8bf9ddc8dd52'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:42.841857 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-6a464030-3dee-401d-9082-8bf9ddc8dd52 admin admin] The token cache is being invalidate because project 0d99d3fd13df429a8ee8a7cb7780adb5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:01:42.846559 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-6a464030-3dee-401d-9082-8bf9ddc8dd52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:42.846559 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-6a464030-3dee-401d-9082-8bf9ddc8dd52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:42.846559 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1162/2326] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:01:42 2022] DELETE /identity/v3/projects/0d99d3fd13df429a8ee8a7cb7780adb5 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:42.850146 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbb6a7-2cc1-4a29-a2d4-cce66e740b19 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:42.850541 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbb6a7-2cc1-4a29-a2d4-cce66e740b19 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:42.850851 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefbb6a7-2cc1-4a29-a2d4-cce66e740b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:42.871147 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-aefbb6a7-2cc1-4a29-a2d4-cce66e740b19 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:42.873210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-aefbb6a7-2cc1-4a29-a2d4-cce66e740b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:42.931600 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1165/2327] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:01:42 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:42.940905 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f92cf-a932-4f86-b321-242605dad9b1 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:42.941450 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f92cf-a932-4f86-b321-242605dad9b1 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:42.941588 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f92cf-a932-4f86-b321-242605dad9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:42.961912 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-9e7f92cf-a932-4f86-b321-242605dad9b1 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:42.963183 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-9e7f92cf-a932-4f86-b321-242605dad9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:42.976864 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1163/2328] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:01:42 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:01:42.981717 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbb6a7-2cc1-4a29-a2d4-cce66e740b19 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:42.992358 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefbb6a7-2cc1-4a29-a2d4-cce66e740b19 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:42.998013 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee8f762-d816-446c-bd97-8ccbf8e36cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:42.998679 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee8f762-d816-446c-bd97-8ccbf8e36cf0 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:42.998959 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee8f762-d816-446c-bd97-8ccbf8e36cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:42.999260 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee8f762-d816-446c-bd97-8ccbf8e36cf0 admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:43.000068 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee8f762-d816-446c-bd97-8ccbf8e36cf0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:01:43.000368 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee8f762-d816-446c-bd97-8ccbf8e36cf0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:43.001993 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee8f762-d816-446c-bd97-8ccbf8e36cf0 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:43.003807 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:43.015924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1166/2329] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 11:01:42 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:01:43.020276 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f92cf-a932-4f86-b321-242605dad9b1 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:43.030405 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f92cf-a932-4f86-b321-242605dad9b1 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:43.035335 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c2951-5421-4968-b24f-0d19e830375c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:43.035956 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c2951-5421-4968-b24f-0d19e830375c admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:43.036233 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c2951-5421-4968-b24f-0d19e830375c admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:43.036522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4c2951-5421-4968-b24f-0d19e830375c admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:43.037337 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c2951-5421-4968-b24f-0d19e830375c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:43.039284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4c2951-5421-4968-b24f-0d19e830375c admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:43.146225 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1164/2330] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:01:43 2022] POST /identity/v3/projects => generated 384 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:01:43.147420 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee8f762-d816-446c-bd97-8ccbf8e36cf0 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:43.158179 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee8f762-d816-446c-bd97-8ccbf8e36cf0 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:43.217919 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783a733-c873-4965-9d61-9864d6d84abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:43.218516 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783a733-c873-4965-9d61-9864d6d84abb admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:43.218795 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783a733-c873-4965-9d61-9864d6d84abb admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:43.219095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783a733-c873-4965-9d61-9864d6d84abb admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:43.219773 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7783a733-c873-4965-9d61-9864d6d84abb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:43.221670 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7783a733-c873-4965-9d61-9864d6d84abb admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:43.321034 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1167/2331] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:01:43 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:01:43.326667 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c2951-5421-4968-b24f-0d19e830375c admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:43.341274 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4c2951-5421-4968-b24f-0d19e830375c admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:43.346418 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b862a-a1af-4b4c-91de-608b7cf33c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:43.347037 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b862a-a1af-4b4c-91de-608b7cf33c3b admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:43.347342 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b862a-a1af-4b4c-91de-608b7cf33c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:43.347616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985b862a-a1af-4b4c-91de-608b7cf33c3b admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:43.348416 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985b862a-a1af-4b4c-91de-608b7cf33c3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:43.350036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985b862a-a1af-4b4c-91de-608b7cf33c3b admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:43.361443 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1165/2332] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:01:43 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:01:43.365484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783a733-c873-4965-9d61-9864d6d84abb admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:43.376720 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783a733-c873-4965-9d61-9864d6d84abb admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:43.381583 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ccf92a-ac38-407d-9124-4be86f05fda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:43.382333 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ccf92a-ac38-407d-9124-4be86f05fda5 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:43.382690 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ccf92a-ac38-407d-9124-4be86f05fda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:43.383066 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ccf92a-ac38-407d-9124-4be86f05fda5 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:43.383836 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ccf92a-ac38-407d-9124-4be86f05fda5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:43.385515 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ccf92a-ac38-407d-9124-4be86f05fda5 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:43.404585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1168/2333] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:01:43 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:01:43.404773 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b862a-a1af-4b4c-91de-608b7cf33c3b admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:43.412344 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985b862a-a1af-4b4c-91de-608b7cf33c3b admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:43.417314 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92de2f6-01b2-4427-b0dd-1e6c754a5b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:43.417991 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92de2f6-01b2-4427-b0dd-1e6c754a5b73 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:43.418772 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92de2f6-01b2-4427-b0dd-1e6c754a5b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:43.419177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92de2f6-01b2-4427-b0dd-1e6c754a5b73 admin admin] PATH_INFO: `/v3/projects/f1ac57e1edbf4fd5b9c988aecc431363/users/79fb99eabdfe47d498da216aacf6f23a/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:43.444348 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92de2f6-01b2-4427-b0dd-1e6c754a5b73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1ac57e1edbf4fd5b9c988aecc431363, user_id=79fb99eabdfe47d498da216aacf6f23a, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:43.450029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92de2f6-01b2-4427-b0dd-1e6c754a5b73 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:43.572503 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1166/2334] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:01:43 2022] PUT /identity/v3/projects/f1ac57e1edbf4fd5b9c988aecc431363/users/79fb99eabdfe47d498da216aacf6f23a/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:43.583264 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389eafe-3162-4345-bb55-8cb58f0edfec None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:43.583869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389eafe-3162-4345-bb55-8cb58f0edfec None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:43.584156 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9389eafe-3162-4345-bb55-8cb58f0edfec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:43.613563 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-9389eafe-3162-4345-bb55-8cb58f0edfec None None] MFA Rules not processed for user `79fb99eabdfe47d498da216aacf6f23a`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:43.616526 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-9389eafe-3162-4345-bb55-8cb58f0edfec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:43.689600 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1169/2335] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:01:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:43.700713 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92de2f6-01b2-4427-b0dd-1e6c754a5b73 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:43.702421 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-b92de2f6-01b2-4427-b0dd-1e6c754a5b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:43.717420 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:43.775626 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92de2f6-01b2-4427-b0dd-1e6c754a5b73 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:43.781695 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbb86cd-675f-4375-8e01-957abd898b1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:43.782307 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb86cd-675f-4375-8e01-957abd898b1b service nova] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:43.782591 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb86cd-675f-4375-8e01-957abd898b1b service nova] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:43.782866 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbb86cd-675f-4375-8e01-957abd898b1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:43.791100 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb86cd-675f-4375-8e01-957abd898b1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:43.792682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbb86cd-675f-4375-8e01-957abd898b1b service nova] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:43.802006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1167/2336] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:01:43 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:01:53.161243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9389eafe-3162-4345-bb55-8cb58f0edfec None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:53.227266 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9389eafe-3162-4345-bb55-8cb58f0edfec None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:53.232631 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a4099-f4cc-446f-88eb-7f885a736db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:53.233308 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a4099-f4cc-446f-88eb-7f885a736db2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:53.233592 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a4099-f4cc-446f-88eb-7f885a736db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:53.233852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a4099-f4cc-446f-88eb-7f885a736db2 admin admin] PATH_INFO: `/v3/users/79fb99eabdfe47d498da216aacf6f23a` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:53.235106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34a4099-f4cc-446f-88eb-7f885a736db2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79fb99eabdfe47d498da216aacf6f23a)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:53.237191 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34a4099-f4cc-446f-88eb-7f885a736db2 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:53.239711 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:53.415799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-a34a4099-f4cc-446f-88eb-7f885a736db2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79fb99eabdfe47d498da216aacf6f23a'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:53.416495 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-a34a4099-f4cc-446f-88eb-7f885a736db2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79fb99eabdfe47d498da216aacf6f23a'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:53.543777 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-a34a4099-f4cc-446f-88eb-7f885a736db2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79fb99eabdfe47d498da216aacf6f23a'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:53.551206 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-a34a4099-f4cc-446f-88eb-7f885a736db2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79fb99eabdfe47d498da216aacf6f23a'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:53.563294 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-a34a4099-f4cc-446f-88eb-7f885a736db2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79fb99eabdfe47d498da216aacf6f23a'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:53.569191 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1170/2337] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:01:53 2022] DELETE /identity/v3/users/79fb99eabdfe47d498da216aacf6f23a => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:53.573632 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51059dce-aad2-4b88-80be-5a8d7180fcbf None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:53.574678 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51059dce-aad2-4b88-80be-5a8d7180fcbf None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:53.574790 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51059dce-aad2-4b88-80be-5a8d7180fcbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:53.594394 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-51059dce-aad2-4b88-80be-5a8d7180fcbf None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:53.595659 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-51059dce-aad2-4b88-80be-5a8d7180fcbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:53.659995 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1168/2338] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:01:53 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:53.670110 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a4099-f4cc-446f-88eb-7f885a736db2 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:53.671568 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-a34a4099-f4cc-446f-88eb-7f885a736db2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:53.729982 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a4099-f4cc-446f-88eb-7f885a736db2 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:53.733772 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af0ee1-44dc-491f-a2ed-614d90ffd0e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:53.734409 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af0ee1-44dc-491f-a2ed-614d90ffd0e3 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:53.734691 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af0ee1-44dc-491f-a2ed-614d90ffd0e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:53.734966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af0ee1-44dc-491f-a2ed-614d90ffd0e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:53.743981 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13af0ee1-44dc-491f-a2ed-614d90ffd0e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:53.745429 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13af0ee1-44dc-491f-a2ed-614d90ffd0e3 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:53.757620 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1171/2339] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:01:53 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:01:54.182598 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51059dce-aad2-4b88-80be-5a8d7180fcbf None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:54.184206 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-51059dce-aad2-4b88-80be-5a8d7180fcbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:54.196394 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51059dce-aad2-4b88-80be-5a8d7180fcbf None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:54.201098 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe824d-2ef1-4655-a522-b9ccc93389c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:54.201745 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe824d-2ef1-4655-a522-b9ccc93389c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:54.202016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe824d-2ef1-4655-a522-b9ccc93389c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:54.202483 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe824d-2ef1-4655-a522-b9ccc93389c5 admin admin] PATH_INFO: `/v3/projects/f1ac57e1edbf4fd5b9c988aecc431363` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:54.212792 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe824d-2ef1-4655-a522-b9ccc93389c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1ac57e1edbf4fd5b9c988aecc431363)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:54.212792 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe824d-2ef1-4655-a522-b9ccc93389c5 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:54.315694 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-3afe824d-2ef1-4655-a522-b9ccc93389c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1ac57e1edbf4fd5b9c988aecc431363', 'request_id': 'req-3afe824d-2ef1-4655-a522-b9ccc93389c5'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:54.409210 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-3afe824d-2ef1-4655-a522-b9ccc93389c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1ac57e1edbf4fd5b9c988aecc431363', 'request_id': 'req-3afe824d-2ef1-4655-a522-b9ccc93389c5'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:54.423385 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-3afe824d-2ef1-4655-a522-b9ccc93389c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1ac57e1edbf4fd5b9c988aecc431363', 'request_id': 'req-3afe824d-2ef1-4655-a522-b9ccc93389c5'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:54.445859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-3afe824d-2ef1-4655-a522-b9ccc93389c5 admin admin] The token cache is being invalidate because project f1ac57e1edbf4fd5b9c988aecc431363 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:01:54.446169 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-3afe824d-2ef1-4655-a522-b9ccc93389c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:54.447287 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-3afe824d-2ef1-4655-a522-b9ccc93389c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:54.451597 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1169/2340] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:01:54 2022] DELETE /identity/v3/projects/f1ac57e1edbf4fd5b9c988aecc431363 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:54.455674 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af403d-29fa-40b6-b4b7-f464171ae239 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:54.456095 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af403d-29fa-40b6-b4b7-f464171ae239 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:54.456199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3af403d-29fa-40b6-b4b7-f464171ae239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:54.482462 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-c3af403d-29fa-40b6-b4b7-f464171ae239 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:54.483896 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-c3af403d-29fa-40b6-b4b7-f464171ae239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:54.551706 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1172/2341] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:01:54 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:54.563147 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77fdb93-0fa8-4d01-a1e5-1e89c5519a50 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:54.563147 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77fdb93-0fa8-4d01-a1e5-1e89c5519a50 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:54.563147 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77fdb93-0fa8-4d01-a1e5-1e89c5519a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:54.581340 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-c77fdb93-0fa8-4d01-a1e5-1e89c5519a50 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:54.582560 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-c77fdb93-0fa8-4d01-a1e5-1e89c5519a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:54.590945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1170/2342] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:01:54 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:54.595381 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af403d-29fa-40b6-b4b7-f464171ae239 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:54.606254 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3af403d-29fa-40b6-b4b7-f464171ae239 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:54.613014 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb042e-70b6-438d-80b7-c060dfb8ea06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:54.613653 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb042e-70b6-438d-80b7-c060dfb8ea06 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:54.613940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb042e-70b6-438d-80b7-c060dfb8ea06 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:54.614366 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdb042e-70b6-438d-80b7-c060dfb8ea06 admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:54.615114 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdb042e-70b6-438d-80b7-c060dfb8ea06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:01:54.615429 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdb042e-70b6-438d-80b7-c060dfb8ea06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:54.617049 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdb042e-70b6-438d-80b7-c060dfb8ea06 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:54.634293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1173/2343] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 11:01:54 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:01:54.635829 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77fdb93-0fa8-4d01-a1e5-1e89c5519a50 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:54.647781 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77fdb93-0fa8-4d01-a1e5-1e89c5519a50 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:54.654726 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc777adf-8c16-4780-892d-4dc3cbb312e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:54.655408 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc777adf-8c16-4780-892d-4dc3cbb312e6 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:54.655692 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc777adf-8c16-4780-892d-4dc3cbb312e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:54.655969 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc777adf-8c16-4780-892d-4dc3cbb312e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:54.657043 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc777adf-8c16-4780-892d-4dc3cbb312e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:54.659353 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc777adf-8c16-4780-892d-4dc3cbb312e6 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:54.681211 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:54.761945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1171/2344] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:01:54 2022] POST /identity/v3/projects => generated 388 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:01:54.763724 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb042e-70b6-438d-80b7-c060dfb8ea06 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:54.774240 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdb042e-70b6-438d-80b7-c060dfb8ea06 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:54.784235 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:54.835268 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267bc9b-d0ce-41f1-8699-b22be921cafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:54.835810 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267bc9b-d0ce-41f1-8699-b22be921cafd admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:54.836100 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267bc9b-d0ce-41f1-8699-b22be921cafd admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:54.836380 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1267bc9b-d0ce-41f1-8699-b22be921cafd admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:54.837047 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1267bc9b-d0ce-41f1-8699-b22be921cafd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:54.839949 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1267bc9b-d0ce-41f1-8699-b22be921cafd admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:54.987087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1174/2345] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:01:54 2022] POST /identity/v3/users => generated 295 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:01:54.989690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc777adf-8c16-4780-892d-4dc3cbb312e6 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:55.000700 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc777adf-8c16-4780-892d-4dc3cbb312e6 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:55.005300 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6650bb5-54c4-47ff-ac97-b89fdf996d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:55.005914 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6650bb5-54c4-47ff-ac97-b89fdf996d3e admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:55.006196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6650bb5-54c4-47ff-ac97-b89fdf996d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:55.006469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6650bb5-54c4-47ff-ac97-b89fdf996d3e admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:55.007190 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6650bb5-54c4-47ff-ac97-b89fdf996d3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:55.008785 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6650bb5-54c4-47ff-ac97-b89fdf996d3e admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:55.019656 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1172/2346] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:01:54 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:01:55.023689 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267bc9b-d0ce-41f1-8699-b22be921cafd admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:55.034902 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1267bc9b-d0ce-41f1-8699-b22be921cafd admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:55.039593 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b33221-b615-499d-af72-891e389fb9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:55.040201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b33221-b615-499d-af72-891e389fb9b2 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:55.040480 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b33221-b615-499d-af72-891e389fb9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:55.040756 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b33221-b615-499d-af72-891e389fb9b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:55.041471 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b33221-b615-499d-af72-891e389fb9b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:55.043704 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b33221-b615-499d-af72-891e389fb9b2 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:55.054507 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1175/2347] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:01:55 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:01:55.058748 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6650bb5-54c4-47ff-ac97-b89fdf996d3e admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:55.068887 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6650bb5-54c4-47ff-ac97-b89fdf996d3e admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:55.073466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb890e9f-5905-4bf7-8c8d-948d6261269f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:55.074683 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb890e9f-5905-4bf7-8c8d-948d6261269f admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:55.074956 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb890e9f-5905-4bf7-8c8d-948d6261269f admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:55.075259 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb890e9f-5905-4bf7-8c8d-948d6261269f admin admin] PATH_INFO: `/v3/projects/cc106d1d1e8c483dac73ce088697e4c3/users/f8cfc5901ee545169ce6e2c2f78f09a0/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:55.090442 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb890e9f-5905-4bf7-8c8d-948d6261269f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc106d1d1e8c483dac73ce088697e4c3, user_id=f8cfc5901ee545169ce6e2c2f78f09a0, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:55.093050 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb890e9f-5905-4bf7-8c8d-948d6261269f admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:55.167624 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1173/2348] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:01:55 2022] PUT /identity/v3/projects/cc106d1d1e8c483dac73ce088697e4c3/users/f8cfc5901ee545169ce6e2c2f78f09a0/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:55.179375 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610de4bc-74f9-49ff-88ed-80c08f63aa7f None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:55.179710 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610de4bc-74f9-49ff-88ed-80c08f63aa7f None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:55.179939 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610de4bc-74f9-49ff-88ed-80c08f63aa7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:55.214426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-610de4bc-74f9-49ff-88ed-80c08f63aa7f None None] MFA Rules not processed for user `f8cfc5901ee545169ce6e2c2f78f09a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:55.216741 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-610de4bc-74f9-49ff-88ed-80c08f63aa7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:55.290165 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1176/2349] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:01:55 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:55.295973 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb890e9f-5905-4bf7-8c8d-948d6261269f admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:55.365162 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb890e9f-5905-4bf7-8c8d-948d6261269f admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:55.370059 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef348888-1df7-4d5f-8ef2-f589ae6e64d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:55.370758 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef348888-1df7-4d5f-8ef2-f589ae6e64d1 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:55.371673 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef348888-1df7-4d5f-8ef2-f589ae6e64d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:55.372051 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef348888-1df7-4d5f-8ef2-f589ae6e64d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:55.372848 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef348888-1df7-4d5f-8ef2-f589ae6e64d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:55.374893 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef348888-1df7-4d5f-8ef2-f589ae6e64d1 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:55.484546 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1174/2350] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:01:55 2022] POST /identity/v3/projects => generated 390 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:01:55.488146 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610de4bc-74f9-49ff-88ed-80c08f63aa7f None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:55.507403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610de4bc-74f9-49ff-88ed-80c08f63aa7f None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:55.608144 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe92a1-96b6-4a5e-ba60-8b5a1e855f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:55.615107 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe92a1-96b6-4a5e-ba60-8b5a1e855f80 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:55.618612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe92a1-96b6-4a5e-ba60-8b5a1e855f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:55.618864 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef348888-1df7-4d5f-8ef2-f589ae6e64d1 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:55.619403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe92a1-96b6-4a5e-ba60-8b5a1e855f80 admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:55.619403 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe92a1-96b6-4a5e-ba60-8b5a1e855f80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:55.619588 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-ef348888-1df7-4d5f-8ef2-f589ae6e64d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:55.619726 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe92a1-96b6-4a5e-ba60-8b5a1e855f80 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:55.708215 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1177/2351] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:01:55 2022] POST /identity/v3/users => generated 302 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:01:55.710261 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef348888-1df7-4d5f-8ef2-f589ae6e64d1 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:55.712428 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe92a1-96b6-4a5e-ba60-8b5a1e855f80 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:55.723569 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199eda3-9b1d-4c35-b957-180d63d22fd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:55.724546 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199eda3-9b1d-4c35-b957-180d63d22fd1 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:55.724780 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe92a1-96b6-4a5e-ba60-8b5a1e855f80 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:55.725205 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199eda3-9b1d-4c35-b957-180d63d22fd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:55.729637 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f199eda3-9b1d-4c35-b957-180d63d22fd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:55.729637 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f199eda3-9b1d-4c35-b957-180d63d22fd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:55.732338 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba3b28-a4bc-4337-9b29-27183f9e397f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:55.733077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba3b28-a4bc-4337-9b29-27183f9e397f admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:55.733363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba3b28-a4bc-4337-9b29-27183f9e397f admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:55.733671 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ba3b28-a4bc-4337-9b29-27183f9e397f admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:55.734491 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba3b28-a4bc-4337-9b29-27183f9e397f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:55.736833 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ba3b28-a4bc-4337-9b29-27183f9e397f admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:55.739426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:55.744716 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f199eda3-9b1d-4c35-b957-180d63d22fd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:55.747198 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f199eda3-9b1d-4c35-b957-180d63d22fd1 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:55.753082 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1178/2352] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:01:55 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:01:55.754264 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba3b28-a4bc-4337-9b29-27183f9e397f admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:55.771886 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ba3b28-a4bc-4337-9b29-27183f9e397f admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:55.780367 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f55e95-4ee4-4750-8e7f-35539bf47f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:55.780971 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f55e95-4ee4-4750-8e7f-35539bf47f29 admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:55.781356 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f55e95-4ee4-4750-8e7f-35539bf47f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:55.781828 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f55e95-4ee4-4750-8e7f-35539bf47f29 admin admin] PATH_INFO: `/v3/projects/5754e15789034a18afec6261f42bfe58/users/5a1a8bda62694384bf769ec161155044/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:55.795285 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1175/2353] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:01:55 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:01:55.801018 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f55e95-4ee4-4750-8e7f-35539bf47f29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5754e15789034a18afec6261f42bfe58, user_id=5a1a8bda62694384bf769ec161155044, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:55.813311 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f55e95-4ee4-4750-8e7f-35539bf47f29 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:55.821135 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199eda3-9b1d-4c35-b957-180d63d22fd1 service neutron] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:55.822902 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f199eda3-9b1d-4c35-b957-180d63d22fd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:55.836978 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f199eda3-9b1d-4c35-b957-180d63d22fd1 service neutron] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:55.841735 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd86d9-c1d3-41ad-8cca-c6380edf1b5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:55.842617 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd86d9-c1d3-41ad-8cca-c6380edf1b5e service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:55.842902 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd86d9-c1d3-41ad-8cca-c6380edf1b5e service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:55.843216 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafd86d9-c1d3-41ad-8cca-c6380edf1b5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:55.845223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-cafd86d9-c1d3-41ad-8cca-c6380edf1b5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:55.860830 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafd86d9-c1d3-41ad-8cca-c6380edf1b5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:55.862497 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafd86d9-c1d3-41ad-8cca-c6380edf1b5e service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:55.873924 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1176/2354] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:01:55 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:01:55.900282 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1179/2355] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:01:55 2022] PUT /identity/v3/projects/5754e15789034a18afec6261f42bfe58/users/5a1a8bda62694384bf769ec161155044/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:55.905153 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd86d9-c1d3-41ad-8cca-c6380edf1b5e service neutron] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:55.919849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafd86d9-c1d3-41ad-8cca-c6380edf1b5e service neutron] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:55.967649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:56.009765 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da3b614-7623-4a1f-a436-869fd4517f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:56.012780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3b614-7623-4a1f-a436-869fd4517f5b admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:56.013207 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3b614-7623-4a1f-a436-869fd4517f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:56.013596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da3b614-7623-4a1f-a436-869fd4517f5b admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:56.014560 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da3b614-7623-4a1f-a436-869fd4517f5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:56.016600 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da3b614-7623-4a1f-a436-869fd4517f5b admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:56.028651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1177/2356] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:01:55 2022] GET /identity/v3/roles => generated 1427 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:01:56.032687 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f55e95-4ee4-4750-8e7f-35539bf47f29 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:56.043857 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f55e95-4ee4-4750-8e7f-35539bf47f29 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:56.054977 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120eb53-e7a9-455f-9826-bf54c9fd4495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:56.057610 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120eb53-e7a9-455f-9826-bf54c9fd4495 admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:56.058276 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120eb53-e7a9-455f-9826-bf54c9fd4495 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:56.059126 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1120eb53-e7a9-455f-9826-bf54c9fd4495 admin admin] PATH_INFO: `/v3/projects/5754e15789034a18afec6261f42bfe58/users/5a1a8bda62694384bf769ec161155044/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:56.063804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1120eb53-e7a9-455f-9826-bf54c9fd4495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5754e15789034a18afec6261f42bfe58, user_id=5a1a8bda62694384bf769ec161155044, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:56.065409 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1120eb53-e7a9-455f-9826-bf54c9fd4495 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:56.083300 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1180/2357] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:01:56 2022] PUT /identity/v3/projects/5754e15789034a18afec6261f42bfe58/users/5a1a8bda62694384bf769ec161155044/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:56.102251 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eaf5e4-c58d-48cb-8551-f3e8c4fb52b8 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:56.102801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eaf5e4-c58d-48cb-8551-f3e8c4fb52b8 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:56.103307 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81eaf5e4-c58d-48cb-8551-f3e8c4fb52b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:56.163888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-81eaf5e4-c58d-48cb-8551-f3e8c4fb52b8 None None] MFA Rules not processed for user `5a1a8bda62694384bf769ec161155044`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:56.166350 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-81eaf5e4-c58d-48cb-8551-f3e8c4fb52b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:56.279999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1178/2358] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:01:56 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:56.300703 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120eb53-e7a9-455f-9826-bf54c9fd4495 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:56.305799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-1120eb53-e7a9-455f-9826-bf54c9fd4495 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:56.407715 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1120eb53-e7a9-455f-9826-bf54c9fd4495 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:56.413039 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee531cc-16dd-4d1f-9a5a-03c3341b6702 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:56.413801 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee531cc-16dd-4d1f-9a5a-03c3341b6702 service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:56.414246 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee531cc-16dd-4d1f-9a5a-03c3341b6702 service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:56.414508 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee531cc-16dd-4d1f-9a5a-03c3341b6702 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:56.426548 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee531cc-16dd-4d1f-9a5a-03c3341b6702 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:56.430394 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee531cc-16dd-4d1f-9a5a-03c3341b6702 service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:56.463735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1181/2359] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:01:56 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:01:56.624913 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eaf5e4-c58d-48cb-8551-f3e8c4fb52b8 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:56.712246 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81eaf5e4-c58d-48cb-8551-f3e8c4fb52b8 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:56.717588 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987f4b2-4636-45d8-a287-0478671d9e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:56.718253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987f4b2-4636-45d8-a287-0478671d9e02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:56.718659 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987f4b2-4636-45d8-a287-0478671d9e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:56.718886 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987f4b2-4636-45d8-a287-0478671d9e02 admin admin] PATH_INFO: `/v3/users/f8cfc5901ee545169ce6e2c2f78f09a0` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:56.720220 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d987f4b2-4636-45d8-a287-0478671d9e02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8cfc5901ee545169ce6e2c2f78f09a0)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:56.722217 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d987f4b2-4636-45d8-a287-0478671d9e02 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:56.987141 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-d987f4b2-4636-45d8-a287-0478671d9e02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8cfc5901ee545169ce6e2c2f78f09a0'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:57.017339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-d987f4b2-4636-45d8-a287-0478671d9e02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8cfc5901ee545169ce6e2c2f78f09a0'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:57.020129 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-d987f4b2-4636-45d8-a287-0478671d9e02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8cfc5901ee545169ce6e2c2f78f09a0'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:57.024171 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:57.129537 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-d987f4b2-4636-45d8-a287-0478671d9e02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8cfc5901ee545169ce6e2c2f78f09a0'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:57.153820 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-d987f4b2-4636-45d8-a287-0478671d9e02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8cfc5901ee545169ce6e2c2f78f09a0'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:57.168745 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1179/2360] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:01:56 2022] DELETE /identity/v3/users/f8cfc5901ee545169ce6e2c2f78f09a0 => generated 0 bytes in 542 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:57.171945 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38383c92-a37d-4502-8257-12652ae69b91 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:57.172469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38383c92-a37d-4502-8257-12652ae69b91 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:57.172755 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38383c92-a37d-4502-8257-12652ae69b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:57.208046 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-38383c92-a37d-4502-8257-12652ae69b91 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:01:57.209582 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-38383c92-a37d-4502-8257-12652ae69b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:57.309453 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1182/2361] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:01:57 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:01:57.330233 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987f4b2-4636-45d8-a287-0478671d9e02 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:57.334369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-d987f4b2-4636-45d8-a287-0478671d9e02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:57.430204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987f4b2-4636-45d8-a287-0478671d9e02 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:57.436255 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49453a-46e6-408f-b4a5-2639b5598e1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:57.439589 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49453a-46e6-408f-b4a5-2639b5598e1e service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:57.439885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49453a-46e6-408f-b4a5-2639b5598e1e service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:57.440226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b49453a-46e6-408f-b4a5-2639b5598e1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:57.459786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b49453a-46e6-408f-b4a5-2639b5598e1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:57.461477 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b49453a-46e6-408f-b4a5-2639b5598e1e service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:57.476240 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1180/2362] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:01:57 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:01:58.476472 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38383c92-a37d-4502-8257-12652ae69b91 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:58.478754 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-38383c92-a37d-4502-8257-12652ae69b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:01:58.498909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38383c92-a37d-4502-8257-12652ae69b91 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:58.519703 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293366a-10fe-4c3b-8add-9fcb7d41d8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:58.520767 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293366a-10fe-4c3b-8add-9fcb7d41d8e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:58.524895 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293366a-10fe-4c3b-8add-9fcb7d41d8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:58.524895 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a293366a-10fe-4c3b-8add-9fcb7d41d8e0 admin admin] PATH_INFO: `/v3/projects/cc106d1d1e8c483dac73ce088697e4c3` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:58.525857 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293366a-10fe-4c3b-8add-9fcb7d41d8e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc106d1d1e8c483dac73ce088697e4c3)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:58.528049 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a293366a-10fe-4c3b-8add-9fcb7d41d8e0 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:58.713688 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-a293366a-10fe-4c3b-8add-9fcb7d41d8e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc106d1d1e8c483dac73ce088697e4c3', 'request_id': 'req-a293366a-10fe-4c3b-8add-9fcb7d41d8e0'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:58.737035 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-a293366a-10fe-4c3b-8add-9fcb7d41d8e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc106d1d1e8c483dac73ce088697e4c3', 'request_id': 'req-a293366a-10fe-4c3b-8add-9fcb7d41d8e0'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:58.746704 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:58.837739 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-a293366a-10fe-4c3b-8add-9fcb7d41d8e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc106d1d1e8c483dac73ce088697e4c3', 'request_id': 'req-a293366a-10fe-4c3b-8add-9fcb7d41d8e0'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:58.856948 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-a293366a-10fe-4c3b-8add-9fcb7d41d8e0 admin admin] The token cache is being invalidate because project cc106d1d1e8c483dac73ce088697e4c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:01:58.857771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-a293366a-10fe-4c3b-8add-9fcb7d41d8e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:58.861636 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-a293366a-10fe-4c3b-8add-9fcb7d41d8e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:58.865204 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1183/2363] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:01:58 2022] DELETE /identity/v3/projects/cc106d1d1e8c483dac73ce088697e4c3 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:01:58.869312 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49453a-46e6-408f-b4a5-2639b5598e1e service neutron] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:01:58.899203 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b49453a-46e6-408f-b4a5-2639b5598e1e service neutron] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:01:59.009945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baff4bb-bbf8-4306-a28f-98ffd2aec7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:01:59.010762 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baff4bb-bbf8-4306-a28f-98ffd2aec7e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:01:59.011270 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baff4bb-bbf8-4306-a28f-98ffd2aec7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:01:59.011993 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baff4bb-bbf8-4306-a28f-98ffd2aec7e4 admin admin] PATH_INFO: `/v3/users/5a1a8bda62694384bf769ec161155044` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:01:59.013452 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baff4bb-bbf8-4306-a28f-98ffd2aec7e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a1a8bda62694384bf769ec161155044)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:01:59.017166 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baff4bb-bbf8-4306-a28f-98ffd2aec7e4 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:01:59.271848 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-7baff4bb-bbf8-4306-a28f-98ffd2aec7e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a1a8bda62694384bf769ec161155044'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:59.280850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-7baff4bb-bbf8-4306-a28f-98ffd2aec7e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a1a8bda62694384bf769ec161155044'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:59.281613 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-7baff4bb-bbf8-4306-a28f-98ffd2aec7e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a1a8bda62694384bf769ec161155044'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:59.378348 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-7baff4bb-bbf8-4306-a28f-98ffd2aec7e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a1a8bda62694384bf769ec161155044'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:59.403846 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:01:59.404942 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-7baff4bb-bbf8-4306-a28f-98ffd2aec7e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a1a8bda62694384bf769ec161155044'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:01:59.413956 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1181/2364] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:01:58 2022] DELETE /identity/v3/users/5a1a8bda62694384bf769ec161155044 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:00.107187 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293366a-10fe-4c3b-8add-9fcb7d41d8e0 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:00.108668 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-a293366a-10fe-4c3b-8add-9fcb7d41d8e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:00.126560 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a293366a-10fe-4c3b-8add-9fcb7d41d8e0 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:00.271682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4833ca39-1a77-4dc0-ae19-273dd72de1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:00.273233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833ca39-1a77-4dc0-ae19-273dd72de1fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:00.274639 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833ca39-1a77-4dc0-ae19-273dd72de1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:00.275778 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4833ca39-1a77-4dc0-ae19-273dd72de1fc admin admin] PATH_INFO: `/v3/projects/5754e15789034a18afec6261f42bfe58` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:00.279656 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833ca39-1a77-4dc0-ae19-273dd72de1fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5754e15789034a18afec6261f42bfe58)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:00.290219 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4833ca39-1a77-4dc0-ae19-273dd72de1fc admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:00.482160 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-4833ca39-1a77-4dc0-ae19-273dd72de1fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5754e15789034a18afec6261f42bfe58', 'request_id': 'req-4833ca39-1a77-4dc0-ae19-273dd72de1fc'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:00.492646 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-4833ca39-1a77-4dc0-ae19-273dd72de1fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5754e15789034a18afec6261f42bfe58', 'request_id': 'req-4833ca39-1a77-4dc0-ae19-273dd72de1fc'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:00.578896 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-4833ca39-1a77-4dc0-ae19-273dd72de1fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5754e15789034a18afec6261f42bfe58', 'request_id': 'req-4833ca39-1a77-4dc0-ae19-273dd72de1fc'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:00.609708 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-4833ca39-1a77-4dc0-ae19-273dd72de1fc admin admin] The token cache is being invalidate because project 5754e15789034a18afec6261f42bfe58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:02:00.615274 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-4833ca39-1a77-4dc0-ae19-273dd72de1fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:00.617574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-4833ca39-1a77-4dc0-ae19-273dd72de1fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:00.626159 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1184/2365] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:02:00 2022] DELETE /identity/v3/projects/5754e15789034a18afec6261f42bfe58 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:00.628700 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed33bf-67da-4a23-9cbb-f1a8b48a3869 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:00.629168 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed33bf-67da-4a23-9cbb-f1a8b48a3869 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:00.629267 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ed33bf-67da-4a23-9cbb-f1a8b48a3869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:00.673703 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-79ed33bf-67da-4a23-9cbb-f1a8b48a3869 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:00.675442 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-79ed33bf-67da-4a23-9cbb-f1a8b48a3869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:00.771032 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1182/2366] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:00 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:00.781664 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd998c1-17dd-4b35-9687-fa486481fa1e None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:00.782475 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd998c1-17dd-4b35-9687-fa486481fa1e None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:00.783226 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd998c1-17dd-4b35-9687-fa486481fa1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:00.812556 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-ebd998c1-17dd-4b35-9687-fa486481fa1e None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:00.813814 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-ebd998c1-17dd-4b35-9687-fa486481fa1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:00.827580 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1185/2367] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:00 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:00.828316 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed33bf-67da-4a23-9cbb-f1a8b48a3869 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:00.845634 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ed33bf-67da-4a23-9cbb-f1a8b48a3869 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:00.853153 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc43460-a672-4511-a162-dfe27b066701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:00.854180 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc43460-a672-4511-a162-dfe27b066701 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:00.854693 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc43460-a672-4511-a162-dfe27b066701 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:00.856673 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc43460-a672-4511-a162-dfe27b066701 admin admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:00.857700 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc43460-a672-4511-a162-dfe27b066701 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:02:00.858219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc43460-a672-4511-a162-dfe27b066701 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:00.860775 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc43460-a672-4511-a162-dfe27b066701 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:00.882393 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1183/2368] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 11:02:00 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:02:00.891740 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd998c1-17dd-4b35-9687-fa486481fa1e None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:00.900101 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd998c1-17dd-4b35-9687-fa486481fa1e None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:00.906683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf437c7-6ee2-4cbd-900d-3fe96934f529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:00.907570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf437c7-6ee2-4cbd-900d-3fe96934f529 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:00.907882 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf437c7-6ee2-4cbd-900d-3fe96934f529 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:00.908134 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf437c7-6ee2-4cbd-900d-3fe96934f529 admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:00.908835 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf437c7-6ee2-4cbd-900d-3fe96934f529 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:00.910934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf437c7-6ee2-4cbd-900d-3fe96934f529 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:01.016345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc43460-a672-4511-a162-dfe27b066701 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:01.016789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1186/2369] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:02:00 2022] POST /identity/v3/projects => generated 386 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:01.031380 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc43460-a672-4511-a162-dfe27b066701 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:01.135157 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ba984-d9db-4e82-b46e-4d31f19ed43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:01.136738 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ba984-d9db-4e82-b46e-4d31f19ed43e admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:01.137247 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ba984-d9db-4e82-b46e-4d31f19ed43e admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:01.138576 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ba984-d9db-4e82-b46e-4d31f19ed43e admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:01.139484 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ba984-d9db-4e82-b46e-4d31f19ed43e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:01.142744 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ba984-d9db-4e82-b46e-4d31f19ed43e admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:01.273217 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1184/2370] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:02:01 2022] POST /identity/v3/users => generated 294 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:02:01.279112 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf437c7-6ee2-4cbd-900d-3fe96934f529 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:01.307743 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf437c7-6ee2-4cbd-900d-3fe96934f529 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:01.326455 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2283d-98dc-43cf-89d4-eb8a8dad1d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:01.327235 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2283d-98dc-43cf-89d4-eb8a8dad1d3e admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:01.327588 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2283d-98dc-43cf-89d4-eb8a8dad1d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:01.328127 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2283d-98dc-43cf-89d4-eb8a8dad1d3e admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:01.328923 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c2283d-98dc-43cf-89d4-eb8a8dad1d3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:01.330580 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c2283d-98dc-43cf-89d4-eb8a8dad1d3e admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:01.356061 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1187/2371] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:01 2022] GET /identity/v3/roles => generated 1427 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:01.356303 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ba984-d9db-4e82-b46e-4d31f19ed43e admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:01.368508 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ba984-d9db-4e82-b46e-4d31f19ed43e admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:01.377314 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f772491-ca29-4126-9491-b08262d2b468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:01.378073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f772491-ca29-4126-9491-b08262d2b468 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:01.378490 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f772491-ca29-4126-9491-b08262d2b468 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:01.378863 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f772491-ca29-4126-9491-b08262d2b468 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:01.380244 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f772491-ca29-4126-9491-b08262d2b468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:01.381775 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f772491-ca29-4126-9491-b08262d2b468 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:01.405718 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1185/2372] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:01 2022] GET /identity/v3/roles => generated 1427 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:02:01.417267 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2283d-98dc-43cf-89d4-eb8a8dad1d3e admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:01.419450 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2283d-98dc-43cf-89d4-eb8a8dad1d3e admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:01.435132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5501cb-2143-4257-83b1-f083a255583f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:01.435132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5501cb-2143-4257-83b1-f083a255583f admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:01.435132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5501cb-2143-4257-83b1-f083a255583f admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:01.435132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5501cb-2143-4257-83b1-f083a255583f admin admin] PATH_INFO: `/v3/projects/f022e65418144d8b9e205985cfba0df7/users/dd2ddb6627824799a2c1831fce7ac1ed/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:01.459821 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5501cb-2143-4257-83b1-f083a255583f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f022e65418144d8b9e205985cfba0df7, user_id=dd2ddb6627824799a2c1831fce7ac1ed, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:01.462795 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5501cb-2143-4257-83b1-f083a255583f admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:01.487484 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f772491-ca29-4126-9491-b08262d2b468 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:01.491789 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-4f772491-ca29-4126-9491-b08262d2b468 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:01.558391 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1188/2373] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:01 2022] PUT /identity/v3/projects/f022e65418144d8b9e205985cfba0df7/users/dd2ddb6627824799a2c1831fce7ac1ed/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:01.560300 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5501cb-2143-4257-83b1-f083a255583f admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:01.563978 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-5f5501cb-2143-4257-83b1-f083a255583f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:01.571676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f772491-ca29-4126-9491-b08262d2b468 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:01.588298 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:01.592215 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:01.639829 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e6d2c5-6d74-470d-bdf7-1feeb8698236 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:01.640589 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6d2c5-6d74-470d-bdf7-1feeb8698236 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:01.640880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6d2c5-6d74-470d-bdf7-1feeb8698236 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:01.641150 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e6d2c5-6d74-470d-bdf7-1feeb8698236 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:01.642982 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-31e6d2c5-6d74-470d-bdf7-1feeb8698236 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:01.657825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5501cb-2143-4257-83b1-f083a255583f admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:01.663441 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c2cbfc-0a59-408f-bb07-a32da959b756 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc4b962e2a742b08beef7187cc527e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:01.664079 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2cbfc-0a59-408f-bb07-a32da959b756 service placement] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:01.664557 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2cbfc-0a59-408f-bb07-a32da959b756 service placement] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:01.665216 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c2cbfc-0a59-408f-bb07-a32da959b756 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:01.667884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-c5c2cbfc-0a59-408f-bb07-a32da959b756 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:01.680034 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c2cbfc-0a59-408f-bb07-a32da959b756 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:01.681747 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c2cbfc-0a59-408f-bb07-a32da959b756 service placement] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:01.692441 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1189/2374] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:01 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:02:01.693950 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea57af-3ba9-4bba-889a-a9c8729e5033 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:01.694448 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea57af-3ba9-4bba-889a-a9c8729e5033 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:01.694729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea57af-3ba9-4bba-889a-a9c8729e5033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:01.729543 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-20ea57af-3ba9-4bba-889a-a9c8729e5033 None None] MFA Rules not processed for user `dd2ddb6627824799a2c1831fce7ac1ed`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:01.732164 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-20ea57af-3ba9-4bba-889a-a9c8729e5033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:01.737212 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e6d2c5-6d74-470d-bdf7-1feeb8698236 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:01.738754 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e6d2c5-6d74-470d-bdf7-1feeb8698236 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:01.750235 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1186/2375] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:01 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:02:01.751376 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e6d2c5-6d74-470d-bdf7-1feeb8698236 service neutron] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:01.752780 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-31e6d2c5-6d74-470d-bdf7-1feeb8698236 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:01.769648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e6d2c5-6d74-470d-bdf7-1feeb8698236 service neutron] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:01.780022 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ba06a-26bd-44ef-9016-340578e85a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:01.786132 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ba06a-26bd-44ef-9016-340578e85a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:01.786132 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ba06a-26bd-44ef-9016-340578e85a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:01.786132 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164ba06a-26bd-44ef-9016-340578e85a8f admin admin] PATH_INFO: `/v3/users/f3f41f17f5154e01a699043edc350cf8` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:01.786132 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164ba06a-26bd-44ef-9016-340578e85a8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3f41f17f5154e01a699043edc350cf8)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:01.788475 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164ba06a-26bd-44ef-9016-340578e85a8f admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:01.826035 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1190/2376] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:01 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:01.827346 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea57af-3ba9-4bba-889a-a9c8729e5033 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:01.830080 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-20ea57af-3ba9-4bba-889a-a9c8729e5033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:01.846237 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea57af-3ba9-4bba-889a-a9c8729e5033 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:01.851686 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3cde4-0ddf-4d2e-8c88-80ab26bdf816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:01.852736 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3cde4-0ddf-4d2e-8c88-80ab26bdf816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:01.852736 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3cde4-0ddf-4d2e-8c88-80ab26bdf816 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:01.852957 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e3cde4-0ddf-4d2e-8c88-80ab26bdf816 admin admin] PATH_INFO: `/v3/users/53bc14ada1564911b2f0a698aef3f333` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:01.854258 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3cde4-0ddf-4d2e-8c88-80ab26bdf816 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53bc14ada1564911b2f0a698aef3f333)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:01.856610 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e3cde4-0ddf-4d2e-8c88-80ab26bdf816 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:02.093774 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-164ba06a-26bd-44ef-9016-340578e85a8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3f41f17f5154e01a699043edc350cf8'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:02.101301 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-164ba06a-26bd-44ef-9016-340578e85a8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3f41f17f5154e01a699043edc350cf8'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:02.102039 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-164ba06a-26bd-44ef-9016-340578e85a8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3f41f17f5154e01a699043edc350cf8'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:02.195267 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-164ba06a-26bd-44ef-9016-340578e85a8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3f41f17f5154e01a699043edc350cf8'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:02.213056 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-92e3cde4-0ddf-4d2e-8c88-80ab26bdf816 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53bc14ada1564911b2f0a698aef3f333'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:02.213305 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-164ba06a-26bd-44ef-9016-340578e85a8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3f41f17f5154e01a699043edc350cf8'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:02.214367 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-92e3cde4-0ddf-4d2e-8c88-80ab26bdf816 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53bc14ada1564911b2f0a698aef3f333'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:02.220624 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1187/2377] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:02:01 2022] DELETE /identity/v3/users/f3f41f17f5154e01a699043edc350cf8 => generated 0 bytes in 470 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:02.221937 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ba06a-26bd-44ef-9016-340578e85a8f admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:02.224752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-164ba06a-26bd-44ef-9016-340578e85a8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:02.278795 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:02.281197 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-92e3cde4-0ddf-4d2e-8c88-80ab26bdf816 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53bc14ada1564911b2f0a698aef3f333'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:02.294674 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-92e3cde4-0ddf-4d2e-8c88-80ab26bdf816 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53bc14ada1564911b2f0a698aef3f333'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:02.311605 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-92e3cde4-0ddf-4d2e-8c88-80ab26bdf816 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53bc14ada1564911b2f0a698aef3f333'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:02.316439 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164ba06a-26bd-44ef-9016-340578e85a8f admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:02.320444 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1191/2378] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:02:01 2022] DELETE /identity/v3/users/53bc14ada1564911b2f0a698aef3f333 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:02.327653 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90e657-9c5e-49dc-90f7-060c0dacfcd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:02.328784 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90e657-9c5e-49dc-90f7-060c0dacfcd5 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:02.329395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90e657-9c5e-49dc-90f7-060c0dacfcd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:02.329395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff90e657-9c5e-49dc-90f7-060c0dacfcd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:02.330717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90e657-9c5e-49dc-90f7-060c0dacfcd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:02.333073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff90e657-9c5e-49dc-90f7-060c0dacfcd5 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:02.498389 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1188/2379] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:02:02 2022] POST /identity/v3/projects => generated 386 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:02.502396 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3cde4-0ddf-4d2e-8c88-80ab26bdf816 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:02.505709 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-92e3cde4-0ddf-4d2e-8c88-80ab26bdf816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:02.514038 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:02.560469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90e657-9c5e-49dc-90f7-060c0dacfcd5 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:02.565028 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-ff90e657-9c5e-49dc-90f7-060c0dacfcd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:02.596251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e3cde4-0ddf-4d2e-8c88-80ab26bdf816 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:02.645350 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff90e657-9c5e-49dc-90f7-060c0dacfcd5 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:02.660338 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea33945-3506-4f2f-bfd1-3e0c56fc497a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:02.663491 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea33945-3506-4f2f-bfd1-3e0c56fc497a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:02.664834 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea33945-3506-4f2f-bfd1-3e0c56fc497a admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:02.665641 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea33945-3506-4f2f-bfd1-3e0c56fc497a admin admin] PATH_INFO: `/v3/projects/302e92110a1545f1b84644b8971bb6eb` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:02.669317 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea33945-3506-4f2f-bfd1-3e0c56fc497a admin admin] RBAC: Authorizing `identity:delete_project(project_id=302e92110a1545f1b84644b8971bb6eb)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:02.671579 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea33945-3506-4f2f-bfd1-3e0c56fc497a admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:02.678837 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb8ac1-2ee8-421a-a906-78acaa3ac5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:02.679857 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb8ac1-2ee8-421a-a906-78acaa3ac5db admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:02.679857 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb8ac1-2ee8-421a-a906-78acaa3ac5db admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:02.680249 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb8ac1-2ee8-421a-a906-78acaa3ac5db admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:02.680997 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb8ac1-2ee8-421a-a906-78acaa3ac5db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:02.683943 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb8ac1-2ee8-421a-a906-78acaa3ac5db admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:02.760975 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1192/2380] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:02:02 2022] POST /identity/v3/users => generated 300 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:02:02.761998 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb8ac1-2ee8-421a-a906-78acaa3ac5db admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:02.763791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-26cb8ac1-2ee8-421a-a906-78acaa3ac5db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:02.777019 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb8ac1-2ee8-421a-a906-78acaa3ac5db admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:02.800069 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-7ea33945-3506-4f2f-bfd1-3e0c56fc497a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '302e92110a1545f1b84644b8971bb6eb', 'request_id': 'req-7ea33945-3506-4f2f-bfd1-3e0c56fc497a'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:02.853845 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb714d63-de77-449b-8f71-9f9913dc5d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:02.854495 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb714d63-de77-449b-8f71-9f9913dc5d1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:02.855450 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb714d63-de77-449b-8f71-9f9913dc5d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:02.856514 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb714d63-de77-449b-8f71-9f9913dc5d1a admin admin] PATH_INFO: `/v3/projects/89a4a0cf1852449e8bfc59ddb5a902c1` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:02.857377 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb714d63-de77-449b-8f71-9f9913dc5d1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=89a4a0cf1852449e8bfc59ddb5a902c1)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:02.859406 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb714d63-de77-449b-8f71-9f9913dc5d1a admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:02.904682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-7ea33945-3506-4f2f-bfd1-3e0c56fc497a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '302e92110a1545f1b84644b8971bb6eb', 'request_id': 'req-7ea33945-3506-4f2f-bfd1-3e0c56fc497a'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:02.909553 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-7ea33945-3506-4f2f-bfd1-3e0c56fc497a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '302e92110a1545f1b84644b8971bb6eb', 'request_id': 'req-7ea33945-3506-4f2f-bfd1-3e0c56fc497a'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:02.922807 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-7ea33945-3506-4f2f-bfd1-3e0c56fc497a admin admin] The token cache is being invalidate because project 302e92110a1545f1b84644b8971bb6eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:02:02.923175 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-7ea33945-3506-4f2f-bfd1-3e0c56fc497a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:02.923864 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-7ea33945-3506-4f2f-bfd1-3e0c56fc497a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:02.926822 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1189/2381] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:02:02 2022] DELETE /identity/v3/projects/302e92110a1545f1b84644b8971bb6eb => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:02.927502 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea33945-3506-4f2f-bfd1-3e0c56fc497a admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:02.928853 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-7ea33945-3506-4f2f-bfd1-3e0c56fc497a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:02.942898 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea33945-3506-4f2f-bfd1-3e0c56fc497a admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:02.948047 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabf55e1-12a8-47ea-b93c-a5bcfacc6708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:02.949373 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf55e1-12a8-47ea-b93c-a5bcfacc6708 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:02.949670 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf55e1-12a8-47ea-b93c-a5bcfacc6708 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:02.949948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabf55e1-12a8-47ea-b93c-a5bcfacc6708 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:02.950718 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf55e1-12a8-47ea-b93c-a5bcfacc6708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:02.952417 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabf55e1-12a8-47ea-b93c-a5bcfacc6708 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:02.967314 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1190/2382] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:02 2022] GET /identity/v3/roles => generated 1427 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:02.970254 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b69fb29-c77f-4371-9985-fa5903681ee7 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:02.970254 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b69fb29-c77f-4371-9985-fa5903681ee7 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:02.970254 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b69fb29-c77f-4371-9985-fa5903681ee7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:03.000642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-bb714d63-de77-449b-8f71-9f9913dc5d1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89a4a0cf1852449e8bfc59ddb5a902c1', 'request_id': 'req-bb714d63-de77-449b-8f71-9f9913dc5d1a'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:03.003959 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-3b69fb29-c77f-4371-9985-fa5903681ee7 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:03.005208 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-3b69fb29-c77f-4371-9985-fa5903681ee7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:03.012149 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-bb714d63-de77-449b-8f71-9f9913dc5d1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89a4a0cf1852449e8bfc59ddb5a902c1', 'request_id': 'req-bb714d63-de77-449b-8f71-9f9913dc5d1a'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:03.062886 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:03.075155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-bb714d63-de77-449b-8f71-9f9913dc5d1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89a4a0cf1852449e8bfc59ddb5a902c1', 'request_id': 'req-bb714d63-de77-449b-8f71-9f9913dc5d1a'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:03.086069 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1191/2383] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:02 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:03.086688 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:03.086884 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b69fb29-c77f-4371-9985-fa5903681ee7 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:03.088888 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-bb714d63-de77-449b-8f71-9f9913dc5d1a admin admin] The token cache is being invalidate because project 89a4a0cf1852449e8bfc59ddb5a902c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:02:03.089655 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-bb714d63-de77-449b-8f71-9f9913dc5d1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:03.090633 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-bb714d63-de77-449b-8f71-9f9913dc5d1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:03.092548 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1193/2384] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:02:02 2022] DELETE /identity/v3/projects/89a4a0cf1852449e8bfc59ddb5a902c1 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:03.102399 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e337e-5ddb-4f50-88dc-1cb1a764626f None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:03.102399 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e337e-5ddb-4f50-88dc-1cb1a764626f None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:03.102399 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e337e-5ddb-4f50-88dc-1cb1a764626f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:03.102732 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b69fb29-c77f-4371-9985-fa5903681ee7 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:03.103438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3528f8d0-5dfa-4391-a02c-2c762003b706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:03.104115 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528f8d0-5dfa-4391-a02c-2c762003b706 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:03.104525 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528f8d0-5dfa-4391-a02c-2c762003b706 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:03.105820 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528f8d0-5dfa-4391-a02c-2c762003b706 admin admin] PATH_INFO: `/v3/projects/647135436c91467db9336ff84482b712/users/c04f8f995bde4f3196c7959c199ee885/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:03.118817 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-215e337e-5ddb-4f50-88dc-1cb1a764626f None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:03.120704 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-215e337e-5ddb-4f50-88dc-1cb1a764626f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:03.122771 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3528f8d0-5dfa-4391-a02c-2c762003b706 admin admin] RBAC: Authorizing `identity:create_grant(project_id=647135436c91467db9336ff84482b712, user_id=c04f8f995bde4f3196c7959c199ee885, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:03.127024 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3528f8d0-5dfa-4391-a02c-2c762003b706 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:03.129878 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1194/2385] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:03 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:02:03.132446 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d709e56-6ceb-4997-817c-4140d7ce8470 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:03.132733 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d709e56-6ceb-4997-817c-4140d7ce8470 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:03.133007 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d709e56-6ceb-4997-817c-4140d7ce8470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:03.153645 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-1d709e56-6ceb-4997-817c-4140d7ce8470 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:03.159672 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-1d709e56-6ceb-4997-817c-4140d7ce8470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:03.162724 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1195/2386] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:03 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:03.163565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d709e56-6ceb-4997-817c-4140d7ce8470 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:03.173816 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d709e56-6ceb-4997-817c-4140d7ce8470 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:03.183757 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1192/2387] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:03 2022] PUT /identity/v3/projects/647135436c91467db9336ff84482b712/users/c04f8f995bde4f3196c7959c199ee885/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:03.185873 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1cd9d-8974-4911-bd42-a1932b2f843e None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:03.186225 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1cd9d-8974-4911-bd42-a1932b2f843e None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:03.186509 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1cd9d-8974-4911-bd42-a1932b2f843e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:03.208198 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-71a1cd9d-8974-4911-bd42-a1932b2f843e None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:03.209410 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-71a1cd9d-8974-4911-bd42-a1932b2f843e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:03.247834 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f2bd28-ccb8-44c2-94f3-640a6d72fb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:03.248723 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2bd28-ccb8-44c2-94f3-640a6d72fb23 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:03.249022 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2bd28-ccb8-44c2-94f3-640a6d72fb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:03.249288 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2bd28-ccb8-44c2-94f3-640a6d72fb23 admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:03.250130 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f2bd28-ccb8-44c2-94f3-640a6d72fb23 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:02:03.250447 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f2bd28-ccb8-44c2-94f3-640a6d72fb23 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:03.252269 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f2bd28-ccb8-44c2-94f3-640a6d72fb23 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:03.266696 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1196/2388] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 11:02:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:02:03.267439 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f2bd28-ccb8-44c2-94f3-640a6d72fb23 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:03.273523 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-d8f2bd28-ccb8-44c2-94f3-640a6d72fb23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:03.287269 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f2bd28-ccb8-44c2-94f3-640a6d72fb23 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:03.291368 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1193/2389] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:03 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:03.291807 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1cd9d-8974-4911-bd42-a1932b2f843e None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:03.295471 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51124c57-0c33-474a-8bb7-f97bb64826b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:03.296141 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51124c57-0c33-474a-8bb7-f97bb64826b0 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:03.296433 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51124c57-0c33-474a-8bb7-f97bb64826b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:03.296712 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51124c57-0c33-474a-8bb7-f97bb64826b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:03.297426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51124c57-0c33-474a-8bb7-f97bb64826b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:03.299159 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51124c57-0c33-474a-8bb7-f97bb64826b0 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:03.307318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1cd9d-8974-4911-bd42-a1932b2f843e None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:03.311873 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1197/2390] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:03 2022] GET /identity/v3/roles => generated 1427 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:03.312809 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51124c57-0c33-474a-8bb7-f97bb64826b0 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:03.317348 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7168ee-ca86-4346-a678-18a079ad5fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:03.317973 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7168ee-ca86-4346-a678-18a079ad5fe9 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:03.318248 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7168ee-ca86-4346-a678-18a079ad5fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:03.318518 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7168ee-ca86-4346-a678-18a079ad5fe9 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:03.319338 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7168ee-ca86-4346-a678-18a079ad5fe9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:03.321348 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7168ee-ca86-4346-a678-18a079ad5fe9 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:03.326607 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51124c57-0c33-474a-8bb7-f97bb64826b0 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:03.332029 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6e4b7-db9e-4c21-8c55-5dc24ac391df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:03.336063 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6e4b7-db9e-4c21-8c55-5dc24ac391df admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:03.336063 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6e4b7-db9e-4c21-8c55-5dc24ac391df admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:03.336063 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6e4b7-db9e-4c21-8c55-5dc24ac391df admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:03.336063 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a6e4b7-db9e-4c21-8c55-5dc24ac391df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:02:03.336063 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a6e4b7-db9e-4c21-8c55-5dc24ac391df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:03.337884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a6e4b7-db9e-4c21-8c55-5dc24ac391df admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:03.353115 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1198/2391] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 11:02:03 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:02:03.353115 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6e4b7-db9e-4c21-8c55-5dc24ac391df admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:03.364892 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6e4b7-db9e-4c21-8c55-5dc24ac391df admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:03.369758 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12536b-13be-4c7d-a06e-61002d8a593a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:03.371099 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12536b-13be-4c7d-a06e-61002d8a593a admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:03.371536 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12536b-13be-4c7d-a06e-61002d8a593a admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:03.371838 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d12536b-13be-4c7d-a06e-61002d8a593a admin admin] PATH_INFO: `/v3/projects/647135436c91467db9336ff84482b712/users/c04f8f995bde4f3196c7959c199ee885/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:03.374548 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12536b-13be-4c7d-a06e-61002d8a593a admin admin] RBAC: Authorizing `identity:create_grant(project_id=647135436c91467db9336ff84482b712, user_id=c04f8f995bde4f3196c7959c199ee885, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:03.376732 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d12536b-13be-4c7d-a06e-61002d8a593a admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:03.390230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1199/2392] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:03 2022] PUT /identity/v3/projects/647135436c91467db9336ff84482b712/users/c04f8f995bde4f3196c7959c199ee885/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:03.391092 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12536b-13be-4c7d-a06e-61002d8a593a admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:03.411898 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1194/2393] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:02:03 2022] POST /identity/v3/projects => generated 372 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:03.413102 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86528577-929e-482f-b949-f7df5d40da9a None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:03.413386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86528577-929e-482f-b949-f7df5d40da9a None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:03.413661 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86528577-929e-482f-b949-f7df5d40da9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:03.460894 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-86528577-929e-482f-b949-f7df5d40da9a None None] MFA Rules not processed for user `c04f8f995bde4f3196c7959c199ee885`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:03.462151 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-86528577-929e-482f-b949-f7df5d40da9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:03.511397 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d12536b-13be-4c7d-a06e-61002d8a593a admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:03.541771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:03.568611 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1195/2394] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:03 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:03.570330 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86528577-929e-482f-b949-f7df5d40da9a None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:03.603807 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f248a-f5f3-40b8-b14d-6e0b8beb1904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:03.605119 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f248a-f5f3-40b8-b14d-6e0b8beb1904 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:03.605506 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f248a-f5f3-40b8-b14d-6e0b8beb1904 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:03.605797 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0f248a-f5f3-40b8-b14d-6e0b8beb1904 admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:03.607324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f248a-f5f3-40b8-b14d-6e0b8beb1904 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:03.609049 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:03.612497 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0f248a-f5f3-40b8-b14d-6e0b8beb1904 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:03.651833 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86528577-929e-482f-b949-f7df5d40da9a None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:03.657822 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f9a46-89ca-4f01-b240-627d2e2cd14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:03.657822 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f9a46-89ca-4f01-b240-627d2e2cd14a admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:03.657822 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f9a46-89ca-4f01-b240-627d2e2cd14a admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:03.658164 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324f9a46-89ca-4f01-b240-627d2e2cd14a admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:03.658597 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f9a46-89ca-4f01-b240-627d2e2cd14a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:03.660639 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324f9a46-89ca-4f01-b240-627d2e2cd14a admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:03.719763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1200/2395] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:02:03 2022] POST /identity/v3/projects => generated 366 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:03.720605 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f248a-f5f3-40b8-b14d-6e0b8beb1904 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:03.722179 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-5b0f248a-f5f3-40b8-b14d-6e0b8beb1904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:03.757609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1196/2396] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:02:03 2022] POST /identity/v3/users => generated 287 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:03.759474 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f9a46-89ca-4f01-b240-627d2e2cd14a admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:03.776268 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324f9a46-89ca-4f01-b240-627d2e2cd14a admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:03.823995 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0f248a-f5f3-40b8-b14d-6e0b8beb1904 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:03.832267 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32748e3-2eca-4463-99cc-5d774fbd7f09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:03.834742 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32748e3-2eca-4463-99cc-5d774fbd7f09 service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:03.836651 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32748e3-2eca-4463-99cc-5d774fbd7f09 service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:03.839896 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32748e3-2eca-4463-99cc-5d774fbd7f09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:03.841023 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-f32748e3-2eca-4463-99cc-5d774fbd7f09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:03.855664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e75e4-114e-4fe7-9492-232eb92d290d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:03.856345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e75e4-114e-4fe7-9492-232eb92d290d admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:03.856635 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e75e4-114e-4fe7-9492-232eb92d290d admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:03.856913 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0e75e4-114e-4fe7-9492-232eb92d290d admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:03.858026 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e75e4-114e-4fe7-9492-232eb92d290d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:03.860216 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0e75e4-114e-4fe7-9492-232eb92d290d admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:03.926667 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32748e3-2eca-4463-99cc-5d774fbd7f09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:03.929276 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32748e3-2eca-4463-99cc-5d774fbd7f09 service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:03.940147 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1197/2397] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:02:03 2022] POST /identity/v3/users => generated 284 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:03.941778 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e75e4-114e-4fe7-9492-232eb92d290d admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:03.949907 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1201/2398] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:03 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:02:03.951304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32748e3-2eca-4463-99cc-5d774fbd7f09 service cinder] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:03.970470 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32748e3-2eca-4463-99cc-5d774fbd7f09 service cinder] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:03.977154 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f33d7c-9542-45b9-a60a-f0de2b7b9979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:03.978605 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f33d7c-9542-45b9-a60a-f0de2b7b9979 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:03.979328 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f33d7c-9542-45b9-a60a-f0de2b7b9979 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:03.979328 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f33d7c-9542-45b9-a60a-f0de2b7b9979 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:03.981295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f33d7c-9542-45b9-a60a-f0de2b7b9979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:03.983619 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0e75e4-114e-4fe7-9492-232eb92d290d admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:03.984369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f33d7c-9542-45b9-a60a-f0de2b7b9979 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:04.002861 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e255554c-4f01-4472-886e-79a15abca490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:04.003894 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e255554c-4f01-4472-886e-79a15abca490 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:04.004384 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e255554c-4f01-4472-886e-79a15abca490 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:04.004870 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e255554c-4f01-4472-886e-79a15abca490 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:04.007958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e255554c-4f01-4472-886e-79a15abca490 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:04.008963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1202/2399] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:03 2022] GET /identity/v3/roles => generated 1427 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:04.010373 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f33d7c-9542-45b9-a60a-f0de2b7b9979 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:04.010881 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e255554c-4f01-4472-886e-79a15abca490 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:04.028333 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f33d7c-9542-45b9-a60a-f0de2b7b9979 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:04.036385 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8cc73c-cae7-4487-b4a2-d1159cdbab00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:04.040323 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8cc73c-cae7-4487-b4a2-d1159cdbab00 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:04.040323 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8cc73c-cae7-4487-b4a2-d1159cdbab00 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:04.040323 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8cc73c-cae7-4487-b4a2-d1159cdbab00 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:04.040323 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8cc73c-cae7-4487-b4a2-d1159cdbab00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:04.043183 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8cc73c-cae7-4487-b4a2-d1159cdbab00 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:04.064632 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1198/2400] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:03 2022] GET /identity/v3/roles => generated 1427 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:04.067665 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1203/2401] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:04 2022] GET /identity/v3/roles => generated 1427 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:04.068223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e255554c-4f01-4472-886e-79a15abca490 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:04.072876 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8cc73c-cae7-4487-b4a2-d1159cdbab00 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:04.089724 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8cc73c-cae7-4487-b4a2-d1159cdbab00 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:04.091746 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e255554c-4f01-4472-886e-79a15abca490 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:04.098120 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c869e7-5643-4717-92a4-1af3fa650011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:04.099142 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c869e7-5643-4717-92a4-1af3fa650011 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:04.099610 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c869e7-5643-4717-92a4-1af3fa650011 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:04.100248 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c869e7-5643-4717-92a4-1af3fa650011 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:04.100477 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930da941-35af-4d81-bc6f-436c57d58d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:04.101160 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930da941-35af-4d81-bc6f-436c57d58d5f admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:04.101435 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930da941-35af-4d81-bc6f-436c57d58d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:04.101648 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c869e7-5643-4717-92a4-1af3fa650011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:04.101767 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930da941-35af-4d81-bc6f-436c57d58d5f admin admin] PATH_INFO: `/v3/projects/406fced738034b25aa43e74c74f95af8/users/9e074cc3fb734ed193504bbbe1af941b/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:04.105443 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c869e7-5643-4717-92a4-1af3fa650011 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:04.140690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930da941-35af-4d81-bc6f-436c57d58d5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=406fced738034b25aa43e74c74f95af8, user_id=9e074cc3fb734ed193504bbbe1af941b, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:04.143863 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1204/2402] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:04 2022] GET /identity/v3/roles => generated 1427 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:04.143960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930da941-35af-4d81-bc6f-436c57d58d5f admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:04.144054 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c869e7-5643-4717-92a4-1af3fa650011 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:04.153302 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c869e7-5643-4717-92a4-1af3fa650011 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:04.161027 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a146db8-5254-484c-8c57-daa59221b6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:04.161975 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a146db8-5254-484c-8c57-daa59221b6ed admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:04.162305 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a146db8-5254-484c-8c57-daa59221b6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:04.162585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a146db8-5254-484c-8c57-daa59221b6ed admin admin] PATH_INFO: `/v3/projects/f78e58a1765d4dbe86a966e510f32cc2/users/65e34c097ae74f3281b3f1418d450f0e/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:04.184793 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a146db8-5254-484c-8c57-daa59221b6ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=f78e58a1765d4dbe86a966e510f32cc2, user_id=65e34c097ae74f3281b3f1418d450f0e, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:04.189667 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a146db8-5254-484c-8c57-daa59221b6ed admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:04.230287 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1199/2403] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:04 2022] PUT /identity/v3/projects/406fced738034b25aa43e74c74f95af8/users/9e074cc3fb734ed193504bbbe1af941b/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:04.240952 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca68d21-2e3c-451e-b5b6-b8064a927fca None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:04.241236 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca68d21-2e3c-451e-b5b6-b8064a927fca None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:04.242042 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca68d21-2e3c-451e-b5b6-b8064a927fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:04.275628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-5ca68d21-2e3c-451e-b5b6-b8064a927fca None None] MFA Rules not processed for user `9e074cc3fb734ed193504bbbe1af941b`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:04.276881 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-5ca68d21-2e3c-451e-b5b6-b8064a927fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:04.306488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1205/2404] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:04 2022] PUT /identity/v3/projects/f78e58a1765d4dbe86a966e510f32cc2/users/65e34c097ae74f3281b3f1418d450f0e/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:02:04.318103 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ba7c4-63bc-4c7e-8ac7-e9abc34c9e44 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:04.318392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ba7c4-63bc-4c7e-8ac7-e9abc34c9e44 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:04.318695 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87ba7c4-63bc-4c7e-8ac7-e9abc34c9e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:04.334097 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:04.348736 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:04.358985 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1200/2405] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:02:04.361100 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca68d21-2e3c-451e-b5b6-b8064a927fca None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:04.361658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-c87ba7c4-63bc-4c7e-8ac7-e9abc34c9e44 None None] MFA Rules not processed for user `65e34c097ae74f3281b3f1418d450f0e`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:04.362935 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-c87ba7c4-63bc-4c7e-8ac7-e9abc34c9e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:04.451782 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1206/2406] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:04 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:02:04.458542 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ba7c4-63bc-4c7e-8ac7-e9abc34c9e44 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:04.461499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca68d21-2e3c-451e-b5b6-b8064a927fca None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:04.525379 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb16a94c-77c1-4c3a-94e4-ab57e04d05f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:04.526302 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16a94c-77c1-4c3a-94e4-ab57e04d05f7 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:04.527264 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16a94c-77c1-4c3a-94e4-ab57e04d05f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:04.528005 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb16a94c-77c1-4c3a-94e4-ab57e04d05f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:04.528790 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb16a94c-77c1-4c3a-94e4-ab57e04d05f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:04.531820 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb16a94c-77c1-4c3a-94e4-ab57e04d05f7 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:04.541377 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87ba7c4-63bc-4c7e-8ac7-e9abc34c9e44 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:04.548506 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2531e92-9c75-4b1d-9120-ca5f116325ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:04.550971 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2531e92-9c75-4b1d-9120-ca5f116325ac admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:04.550971 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2531e92-9c75-4b1d-9120-ca5f116325ac admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:04.550971 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2531e92-9c75-4b1d-9120-ca5f116325ac admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:04.551536 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2531e92-9c75-4b1d-9120-ca5f116325ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:04.554468 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2531e92-9c75-4b1d-9120-ca5f116325ac admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:04.674751 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1207/2407] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:02:04 2022] POST /identity/v3/projects => generated 374 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:04.676439 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1201/2408] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:02:04 2022] POST /identity/v3/projects => generated 364 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:04.679452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2531e92-9c75-4b1d-9120-ca5f116325ac admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:04.679627 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb16a94c-77c1-4c3a-94e4-ab57e04d05f7 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:04.747817 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb16a94c-77c1-4c3a-94e4-ab57e04d05f7 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:04.753643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e609a-e952-4ea9-a12d-fa29243cacd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:04.754276 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e609a-e952-4ea9-a12d-fa29243cacd2 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:04.754564 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e609a-e952-4ea9-a12d-fa29243cacd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:04.754837 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619e609a-e952-4ea9-a12d-fa29243cacd2 admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:04.754951 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2531e92-9c75-4b1d-9120-ca5f116325ac admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:04.756212 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e609a-e952-4ea9-a12d-fa29243cacd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:04.758282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619e609a-e952-4ea9-a12d-fa29243cacd2 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:04.760260 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdf7fb8-9395-491f-832e-9d447026ea2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:04.761059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf7fb8-9395-491f-832e-9d447026ea2f admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:04.761345 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf7fb8-9395-491f-832e-9d447026ea2f admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:04.761779 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdf7fb8-9395-491f-832e-9d447026ea2f admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:04.762464 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdf7fb8-9395-491f-832e-9d447026ea2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:04.764423 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdf7fb8-9395-491f-832e-9d447026ea2f admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:04.885586 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1208/2409] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:02:04 2022] POST /identity/v3/users => generated 294 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:02:04.889579 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1202/2410] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:02:04 2022] POST /identity/v3/users => generated 289 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:02:04.892108 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdf7fb8-9395-491f-832e-9d447026ea2f admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:04.895144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e609a-e952-4ea9-a12d-fa29243cacd2 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:04.905911 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdf7fb8-9395-491f-832e-9d447026ea2f admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:04.913811 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d036cad-b503-4a22-95cc-2b073f590ce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:04.914787 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d036cad-b503-4a22-95cc-2b073f590ce2 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:04.915375 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d036cad-b503-4a22-95cc-2b073f590ce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:04.915823 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d036cad-b503-4a22-95cc-2b073f590ce2 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:04.916910 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d036cad-b503-4a22-95cc-2b073f590ce2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:04.917690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619e609a-e952-4ea9-a12d-fa29243cacd2 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:04.919647 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d036cad-b503-4a22-95cc-2b073f590ce2 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:04.922796 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:04.923871 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02e055-4325-4efb-98f7-db140690fa09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:04.924510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02e055-4325-4efb-98f7-db140690fa09 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:04.924784 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02e055-4325-4efb-98f7-db140690fa09 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:04.925056 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe02e055-4325-4efb-98f7-db140690fa09 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:04.925910 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02e055-4325-4efb-98f7-db140690fa09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:04.927369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe02e055-4325-4efb-98f7-db140690fa09 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:04.933386 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1209/2411] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:04 2022] GET /identity/v3/roles => generated 1427 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:02:04.941198 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1203/2412] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:04 2022] GET /identity/v3/roles => generated 1427 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:02:04.946132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d036cad-b503-4a22-95cc-2b073f590ce2 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:04.947597 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02e055-4325-4efb-98f7-db140690fa09 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:04.960831 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d036cad-b503-4a22-95cc-2b073f590ce2 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:04.963119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe02e055-4325-4efb-98f7-db140690fa09 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:04.967773 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4bc81-895a-4845-ae89-eaa4125b1a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:04.968471 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4bc81-895a-4845-ae89-eaa4125b1a51 admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:04.974590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4bc81-895a-4845-ae89-eaa4125b1a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:04.974590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4bc81-895a-4845-ae89-eaa4125b1a51 admin admin] PATH_INFO: `/v3/projects/1f1d31337aaa47d2854acd58e35bc94e/users/e40c68f50a0642bb9e846955fafc827d/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:04.975114 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672180c0-a6cd-4fa6-bb7d-42be4ebe9c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:04.975114 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672180c0-a6cd-4fa6-bb7d-42be4ebe9c60 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:04.975114 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672180c0-a6cd-4fa6-bb7d-42be4ebe9c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:04.975114 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672180c0-a6cd-4fa6-bb7d-42be4ebe9c60 admin admin] PATH_INFO: `/v3/projects/eabf9882be3f46aeb1835813241953eb/users/2ebd867695b541199f02cc754bb05bcc/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:04.992371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:04.997019 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4bc81-895a-4845-ae89-eaa4125b1a51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f1d31337aaa47d2854acd58e35bc94e, user_id=e40c68f50a0642bb9e846955fafc827d, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:05.008854 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4bc81-895a-4845-ae89-eaa4125b1a51 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:05.037923 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672180c0-a6cd-4fa6-bb7d-42be4ebe9c60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eabf9882be3f46aeb1835813241953eb, user_id=2ebd867695b541199f02cc754bb05bcc, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:05.037923 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672180c0-a6cd-4fa6-bb7d-42be4ebe9c60 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:05.096939 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1210/2413] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:04 2022] PUT /identity/v3/projects/1f1d31337aaa47d2854acd58e35bc94e/users/e40c68f50a0642bb9e846955fafc827d/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:02:05.105581 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4bc81-895a-4845-ae89-eaa4125b1a51 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:05.137989 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1204/2414] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:04 2022] PUT /identity/v3/projects/eabf9882be3f46aeb1835813241953eb/users/2ebd867695b541199f02cc754bb05bcc/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:05.142451 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672180c0-a6cd-4fa6-bb7d-42be4ebe9c60 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:05.211990 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4bc81-895a-4845-ae89-eaa4125b1a51 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:05.230594 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672180c0-a6cd-4fa6-bb7d-42be4ebe9c60 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:05.238387 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60ec0b-8d87-422d-bd03-a59ef77f1526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:05.238576 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60ec0b-8d87-422d-bd03-a59ef77f1526 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:05.238950 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60ec0b-8d87-422d-bd03-a59ef77f1526 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:05.239328 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff60ec0b-8d87-422d-bd03-a59ef77f1526 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:05.240097 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60ec0b-8d87-422d-bd03-a59ef77f1526 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:05.241823 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff60ec0b-8d87-422d-bd03-a59ef77f1526 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:05.260281 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1205/2415] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:05 2022] GET /identity/v3/roles => generated 1427 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:02:05.262025 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60ec0b-8d87-422d-bd03-a59ef77f1526 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:05.273285 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff60ec0b-8d87-422d-bd03-a59ef77f1526 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:05.280463 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5cf4d-2333-4257-a28e-92056af5994e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:05.281149 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5cf4d-2333-4257-a28e-92056af5994e admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:05.281476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5cf4d-2333-4257-a28e-92056af5994e admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:05.281753 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf5cf4d-2333-4257-a28e-92056af5994e admin admin] PATH_INFO: `/v3/projects/eabf9882be3f46aeb1835813241953eb/users/2ebd867695b541199f02cc754bb05bcc/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:05.283027 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5785098-ede8-47cf-bcbb-075cbf302e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:05.283838 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5785098-ede8-47cf-bcbb-075cbf302e3e admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:05.284169 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5cf4d-2333-4257-a28e-92056af5994e admin admin] RBAC: Authorizing `identity:create_grant(project_id=eabf9882be3f46aeb1835813241953eb, user_id=2ebd867695b541199f02cc754bb05bcc, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:05.284305 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5785098-ede8-47cf-bcbb-075cbf302e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:05.284583 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5785098-ede8-47cf-bcbb-075cbf302e3e admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:05.285345 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5785098-ede8-47cf-bcbb-075cbf302e3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:05.287319 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5785098-ede8-47cf-bcbb-075cbf302e3e admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:05.287738 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf5cf4d-2333-4257-a28e-92056af5994e admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:05.303095 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1206/2416] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:05 2022] PUT /identity/v3/projects/eabf9882be3f46aeb1835813241953eb/users/2ebd867695b541199f02cc754bb05bcc/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:05.307607 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1211/2417] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:05 2022] GET /identity/v3/roles => generated 1427 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:02:05.318685 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5cf4d-2333-4257-a28e-92056af5994e admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:05.320330 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e789206-c5de-4d53-8269-721e1c26f91d None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:05.320617 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e789206-c5de-4d53-8269-721e1c26f91d None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:05.320895 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e789206-c5de-4d53-8269-721e1c26f91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:05.371005 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-1e789206-c5de-4d53-8269-721e1c26f91d None None] MFA Rules not processed for user `2ebd867695b541199f02cc754bb05bcc`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:05.373503 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-1e789206-c5de-4d53-8269-721e1c26f91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:05.454918 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf5cf4d-2333-4257-a28e-92056af5994e admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:05.462111 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c641b33-3391-4c7a-b932-1918285235c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:05.462963 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c641b33-3391-4c7a-b932-1918285235c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:05.463293 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c641b33-3391-4c7a-b932-1918285235c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:05.463575 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c641b33-3391-4c7a-b932-1918285235c1 admin admin] PATH_INFO: `/v3/projects/1f1d31337aaa47d2854acd58e35bc94e/users/e40c68f50a0642bb9e846955fafc827d/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:05.466235 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c641b33-3391-4c7a-b932-1918285235c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f1d31337aaa47d2854acd58e35bc94e, user_id=e40c68f50a0642bb9e846955fafc827d, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:05.469629 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c641b33-3391-4c7a-b932-1918285235c1 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:05.485430 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1207/2418] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:05 2022] PUT /identity/v3/projects/1f1d31337aaa47d2854acd58e35bc94e/users/e40c68f50a0642bb9e846955fafc827d/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:02:05.498317 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe89e4-a482-44db-9df8-c6382ffabef2 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:05.498664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe89e4-a482-44db-9df8-c6382ffabef2 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:05.502636 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe89e4-a482-44db-9df8-c6382ffabef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:05.528455 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1212/2419] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:05 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:05.529551 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e789206-c5de-4d53-8269-721e1c26f91d None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:05.530974 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-1e789206-c5de-4d53-8269-721e1c26f91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:05.536843 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:05.545306 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-debe89e4-a482-44db-9df8-c6382ffabef2 None None] MFA Rules not processed for user `e40c68f50a0642bb9e846955fafc827d`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:05.546585 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-debe89e4-a482-44db-9df8-c6382ffabef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:05.616730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e789206-c5de-4d53-8269-721e1c26f91d None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:05.620697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:05.649991 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1208/2420] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:05 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:05.662935 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe89e4-a482-44db-9df8-c6382ffabef2 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:05.690958 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a46060-5893-43b5-be17-3d0f6a5775c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:05.691563 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a46060-5893-43b5-be17-3d0f6a5775c5 service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:05.691925 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a46060-5893-43b5-be17-3d0f6a5775c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:05.692219 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a46060-5893-43b5-be17-3d0f6a5775c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:05.729371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe89e4-a482-44db-9df8-c6382ffabef2 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:05.736320 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e2d4c-704d-4e66-9630-b4805027c532 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:05.737292 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e2d4c-704d-4e66-9630-b4805027c532 service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:05.737616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e2d4c-704d-4e66-9630-b4805027c532 service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:05.738817 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335e2d4c-704d-4e66-9630-b4805027c532 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:05.752996 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e2d4c-704d-4e66-9630-b4805027c532 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:05.754494 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335e2d4c-704d-4e66-9630-b4805027c532 service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:05.766018 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1209/2421] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:05 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:02:05.783880 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a46060-5893-43b5-be17-3d0f6a5775c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:05.785334 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a46060-5893-43b5-be17-3d0f6a5775c5 service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:05.810777 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1213/2422] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:02:06.086879 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e2d4c-704d-4e66-9630-b4805027c532 service cinder] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:06.088709 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-335e2d4c-704d-4e66-9630-b4805027c532 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:06.183682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335e2d4c-704d-4e66-9630-b4805027c532 service cinder] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:06.189048 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96580759-9a84-46d5-9504-bbc8b520fc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:06.189895 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96580759-9a84-46d5-9504-bbc8b520fc1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:06.190366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96580759-9a84-46d5-9504-bbc8b520fc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:06.190650 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96580759-9a84-46d5-9504-bbc8b520fc1e admin admin] PATH_INFO: `/v3/users/ab1211acdd9a4df48c46f8807ed00063` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:06.192118 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96580759-9a84-46d5-9504-bbc8b520fc1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab1211acdd9a4df48c46f8807ed00063)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:06.195704 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96580759-9a84-46d5-9504-bbc8b520fc1e admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:06.485600 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-96580759-9a84-46d5-9504-bbc8b520fc1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab1211acdd9a4df48c46f8807ed00063'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:06.492633 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-96580759-9a84-46d5-9504-bbc8b520fc1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab1211acdd9a4df48c46f8807ed00063'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:06.493377 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-96580759-9a84-46d5-9504-bbc8b520fc1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab1211acdd9a4df48c46f8807ed00063'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:06.615157 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-96580759-9a84-46d5-9504-bbc8b520fc1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab1211acdd9a4df48c46f8807ed00063'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:06.628287 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-96580759-9a84-46d5-9504-bbc8b520fc1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab1211acdd9a4df48c46f8807ed00063'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:06.634456 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1210/2423] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:02:06 2022] DELETE /identity/v3/users/ab1211acdd9a4df48c46f8807ed00063 => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:07.179196 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a46060-5893-43b5-be17-3d0f6a5775c5 service cinder] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:07.190841 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a46060-5893-43b5-be17-3d0f6a5775c5 service cinder] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:07.261773 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cddb7ea-9efc-4ae9-bb37-0b1ed4a51fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:07.262416 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cddb7ea-9efc-4ae9-bb37-0b1ed4a51fa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:07.262701 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cddb7ea-9efc-4ae9-bb37-0b1ed4a51fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:07.262977 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cddb7ea-9efc-4ae9-bb37-0b1ed4a51fa5 admin admin] PATH_INFO: `/v3/projects/1b3a294d5d02485980c82f065ee76bd7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:07.264239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cddb7ea-9efc-4ae9-bb37-0b1ed4a51fa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b3a294d5d02485980c82f065ee76bd7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:07.266207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cddb7ea-9efc-4ae9-bb37-0b1ed4a51fa5 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:07.277365 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:07.292604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96580759-9a84-46d5-9504-bbc8b520fc1e admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:07.296573 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-96580759-9a84-46d5-9504-bbc8b520fc1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:07.308489 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96580759-9a84-46d5-9504-bbc8b520fc1e admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:07.313111 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dbf61f-e989-4509-ace3-8f139a38af52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:07.313746 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbf61f-e989-4509-ace3-8f139a38af52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:07.314051 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbf61f-e989-4509-ace3-8f139a38af52 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:07.314326 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dbf61f-e989-4509-ace3-8f139a38af52 admin admin] PATH_INFO: `/v3/users/65e34c097ae74f3281b3f1418d450f0e` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:07.315541 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dbf61f-e989-4509-ace3-8f139a38af52 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65e34c097ae74f3281b3f1418d450f0e)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:07.317539 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dbf61f-e989-4509-ace3-8f139a38af52 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:07.320314 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:07.379033 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-0cddb7ea-9efc-4ae9-bb37-0b1ed4a51fa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b3a294d5d02485980c82f065ee76bd7', 'request_id': 'req-0cddb7ea-9efc-4ae9-bb37-0b1ed4a51fa5'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:07.385396 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-0cddb7ea-9efc-4ae9-bb37-0b1ed4a51fa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b3a294d5d02485980c82f065ee76bd7', 'request_id': 'req-0cddb7ea-9efc-4ae9-bb37-0b1ed4a51fa5'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:07.494460 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-0cddb7ea-9efc-4ae9-bb37-0b1ed4a51fa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b3a294d5d02485980c82f065ee76bd7', 'request_id': 'req-0cddb7ea-9efc-4ae9-bb37-0b1ed4a51fa5'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:07.507042 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-0cddb7ea-9efc-4ae9-bb37-0b1ed4a51fa5 admin admin] The token cache is being invalidate because project 1b3a294d5d02485980c82f065ee76bd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:02:07.507365 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-0cddb7ea-9efc-4ae9-bb37-0b1ed4a51fa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:07.508038 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-0cddb7ea-9efc-4ae9-bb37-0b1ed4a51fa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:07.510001 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1214/2424] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:02:07 2022] DELETE /identity/v3/projects/1b3a294d5d02485980c82f065ee76bd7 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:07.511278 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cddb7ea-9efc-4ae9-bb37-0b1ed4a51fa5 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:07.513106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-0cddb7ea-9efc-4ae9-bb37-0b1ed4a51fa5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:07.524449 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-e2dbf61f-e989-4509-ace3-8f139a38af52 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65e34c097ae74f3281b3f1418d450f0e'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:07.533646 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-e2dbf61f-e989-4509-ace3-8f139a38af52 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65e34c097ae74f3281b3f1418d450f0e'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:07.534350 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-e2dbf61f-e989-4509-ace3-8f139a38af52 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65e34c097ae74f3281b3f1418d450f0e'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:07.594936 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cddb7ea-9efc-4ae9-bb37-0b1ed4a51fa5 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:07.599804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a095b8ca-df43-4b5e-a5d3-25b62220f33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:07.600807 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095b8ca-df43-4b5e-a5d3-25b62220f33e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:07.600807 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095b8ca-df43-4b5e-a5d3-25b62220f33e admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:07.601026 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a095b8ca-df43-4b5e-a5d3-25b62220f33e admin admin] PATH_INFO: `/v3/users/9e074cc3fb734ed193504bbbe1af941b` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:07.602560 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095b8ca-df43-4b5e-a5d3-25b62220f33e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e074cc3fb734ed193504bbbe1af941b)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:07.604257 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a095b8ca-df43-4b5e-a5d3-25b62220f33e admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:07.634996 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-e2dbf61f-e989-4509-ace3-8f139a38af52 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65e34c097ae74f3281b3f1418d450f0e'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:07.647943 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-e2dbf61f-e989-4509-ace3-8f139a38af52 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65e34c097ae74f3281b3f1418d450f0e'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:07.655034 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1211/2425] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:02:07 2022] DELETE /identity/v3/users/65e34c097ae74f3281b3f1418d450f0e => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:07.656358 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420621e8-de78-4030-85c4-fdcabed277b0 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:07.656647 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420621e8-de78-4030-85c4-fdcabed277b0 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:07.656922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420621e8-de78-4030-85c4-fdcabed277b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:07.677301 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-420621e8-de78-4030-85c4-fdcabed277b0 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:07.678684 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-420621e8-de78-4030-85c4-fdcabed277b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:07.740545 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1212/2426] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:07 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:07.741711 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7721103-8f68-4f35-a630-ee4905201175 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:07.741991 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7721103-8f68-4f35-a630-ee4905201175 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:07.742262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7721103-8f68-4f35-a630-ee4905201175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:07.765590 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-d7721103-8f68-4f35-a630-ee4905201175 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:07.767011 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-d7721103-8f68-4f35-a630-ee4905201175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:07.775102 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1213/2427] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:07 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:07.776269 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2498827f-ce37-4bcf-9219-6c0b41b65f9f None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:07.776553 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2498827f-ce37-4bcf-9219-6c0b41b65f9f None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:07.776817 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2498827f-ce37-4bcf-9219-6c0b41b65f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:07.798142 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-2498827f-ce37-4bcf-9219-6c0b41b65f9f None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:07.799362 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-2498827f-ce37-4bcf-9219-6c0b41b65f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:07.807337 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1214/2428] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:07 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:07.808270 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2498827f-ce37-4bcf-9219-6c0b41b65f9f None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:07.809742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-2498827f-ce37-4bcf-9219-6c0b41b65f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:07.857963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-a095b8ca-df43-4b5e-a5d3-25b62220f33e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e074cc3fb734ed193504bbbe1af941b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:07.858788 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-a095b8ca-df43-4b5e-a5d3-25b62220f33e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e074cc3fb734ed193504bbbe1af941b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:07.866842 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2498827f-ce37-4bcf-9219-6c0b41b65f9f None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:07.870584 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2b211-a753-4072-ab17-fe234514a91a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:07.871310 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2b211-a753-4072-ab17-fe234514a91a service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:07.871901 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2b211-a753-4072-ab17-fe234514a91a service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:07.872286 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd2b211-a753-4072-ab17-fe234514a91a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:07.881828 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd2b211-a753-4072-ab17-fe234514a91a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:07.883429 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd2b211-a753-4072-ab17-fe234514a91a service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:07.893318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1215/2429] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:07 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:02:07.894098 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2b211-a753-4072-ab17-fe234514a91a service neutron] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:07.903845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd2b211-a753-4072-ab17-fe234514a91a service neutron] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:07.909473 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96b8bc-dfac-4c54-ab2a-9711f0c43d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:07.910144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96b8bc-dfac-4c54-ab2a-9711f0c43d55 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:07.910659 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96b8bc-dfac-4c54-ab2a-9711f0c43d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:07.910932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96b8bc-dfac-4c54-ab2a-9711f0c43d55 admin admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:07.911653 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b96b8bc-dfac-4c54-ab2a-9711f0c43d55 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:02:07.911954 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b96b8bc-dfac-4c54-ab2a-9711f0c43d55 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:07.913531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b96b8bc-dfac-4c54-ab2a-9711f0c43d55 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:07.921744 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:07.925934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-a095b8ca-df43-4b5e-a5d3-25b62220f33e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e074cc3fb734ed193504bbbe1af941b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:07.927947 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1216/2430] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 11:02:07 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:02:07.932851 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96b8bc-dfac-4c54-ab2a-9711f0c43d55 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:07.934408 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-a095b8ca-df43-4b5e-a5d3-25b62220f33e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e074cc3fb734ed193504bbbe1af941b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:07.943416 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96b8bc-dfac-4c54-ab2a-9711f0c43d55 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:07.948311 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-a095b8ca-df43-4b5e-a5d3-25b62220f33e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e074cc3fb734ed193504bbbe1af941b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:07.949443 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d8f04c-a1f2-4c08-9699-e6538ddce422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:07.950448 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d8f04c-a1f2-4c08-9699-e6538ddce422 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:07.950448 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d8f04c-a1f2-4c08-9699-e6538ddce422 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:07.950750 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d8f04c-a1f2-4c08-9699-e6538ddce422 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:07.951876 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d8f04c-a1f2-4c08-9699-e6538ddce422 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:07.953370 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d8f04c-a1f2-4c08-9699-e6538ddce422 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:07.957120 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1215/2431] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:02:07 2022] DELETE /identity/v3/users/9e074cc3fb734ed193504bbbe1af941b => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:07.961504 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0c8ac-d250-4afc-bcae-c850a09c1373 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:07.962047 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0c8ac-d250-4afc-bcae-c850a09c1373 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:07.962047 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc0c8ac-d250-4afc-bcae-c850a09c1373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:07.983570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-edc0c8ac-d250-4afc-bcae-c850a09c1373 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:07.984734 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-edc0c8ac-d250-4afc-bcae-c850a09c1373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:08.002362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:08.048200 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1217/2432] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:02:07 2022] POST /identity/v3/projects => generated 374 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:02:08.049929 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1216/2433] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:07 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:02:08.054036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d8f04c-a1f2-4c08-9699-e6538ddce422 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:08.055939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-e8d8f04c-a1f2-4c08-9699-e6538ddce422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:08.060835 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0c8ac-d250-4afc-bcae-c850a09c1373 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:08.063575 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-edc0c8ac-d250-4afc-bcae-c850a09c1373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:08.148299 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc0c8ac-d250-4afc-bcae-c850a09c1373 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:08.169118 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d8f04c-a1f2-4c08-9699-e6538ddce422 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:08.175938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d61772-f824-42ce-97f7-d25eea1827bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:08.176782 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d61772-f824-42ce-97f7-d25eea1827bb admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:08.177585 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d61772-f824-42ce-97f7-d25eea1827bb admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:08.178291 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d61772-f824-42ce-97f7-d25eea1827bb admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:08.179076 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d61772-f824-42ce-97f7-d25eea1827bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:08.181463 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d61772-f824-42ce-97f7-d25eea1827bb admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:08.221810 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c968a0a-edea-452e-b921-65b61f3a258f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:08.222542 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c968a0a-edea-452e-b921-65b61f3a258f service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:08.222887 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c968a0a-edea-452e-b921-65b61f3a258f service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:08.223205 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c968a0a-edea-452e-b921-65b61f3a258f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:08.237927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c968a0a-edea-452e-b921-65b61f3a258f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:08.240203 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c968a0a-edea-452e-b921-65b61f3a258f service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:08.256835 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1217/2434] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:08 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:02:08.284515 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1218/2435] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:02:08 2022] POST /identity/v3/users => generated 288 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:08.288903 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c968a0a-edea-452e-b921-65b61f3a258f service neutron] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:08.301408 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c968a0a-edea-452e-b921-65b61f3a258f service neutron] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:08.306247 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6652d92-70a1-4c22-9a97-c61ff0a7f626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:08.307033 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6652d92-70a1-4c22-9a97-c61ff0a7f626 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:08.307262 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6652d92-70a1-4c22-9a97-c61ff0a7f626 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:08.307551 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6652d92-70a1-4c22-9a97-c61ff0a7f626 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:08.308357 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6652d92-70a1-4c22-9a97-c61ff0a7f626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:08.311031 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6652d92-70a1-4c22-9a97-c61ff0a7f626 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:08.326333 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1218/2436] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:08 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:08.327856 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d61772-f824-42ce-97f7-d25eea1827bb admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:08.338167 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d61772-f824-42ce-97f7-d25eea1827bb admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:08.344058 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97155bf5-682f-4ff0-aa4b-a4f3214175b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:08.344878 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97155bf5-682f-4ff0-aa4b-a4f3214175b7 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:08.345241 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97155bf5-682f-4ff0-aa4b-a4f3214175b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:08.345589 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97155bf5-682f-4ff0-aa4b-a4f3214175b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:08.346347 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97155bf5-682f-4ff0-aa4b-a4f3214175b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:08.347991 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97155bf5-682f-4ff0-aa4b-a4f3214175b7 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:08.359322 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1219/2437] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:08 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:08.363581 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6652d92-70a1-4c22-9a97-c61ff0a7f626 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:08.373937 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97155bf5-682f-4ff0-aa4b-a4f3214175b7 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:08.376559 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6652d92-70a1-4c22-9a97-c61ff0a7f626 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:08.378831 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-97155bf5-682f-4ff0-aa4b-a4f3214175b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:08.386804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fb806b-3450-4b0d-a6f2-8aae660a420a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:08.387842 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb806b-3450-4b0d-a6f2-8aae660a420a admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:08.388258 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb806b-3450-4b0d-a6f2-8aae660a420a admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:08.388494 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fb806b-3450-4b0d-a6f2-8aae660a420a admin admin] PATH_INFO: `/v3/projects/74000cc7fc2845628610ca8b17ae395b/users/ab4b8cf9f04940428a8792a09f824378/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:08.410498 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97155bf5-682f-4ff0-aa4b-a4f3214175b7 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:08.417293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb806b-3450-4b0d-a6f2-8aae660a420a admin admin] RBAC: Authorizing `identity:create_grant(project_id=74000cc7fc2845628610ca8b17ae395b, user_id=ab4b8cf9f04940428a8792a09f824378, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:08.417568 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3cf702-16c9-45a4-ac98-7abe14ed0b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:08.418811 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3cf702-16c9-45a4-ac98-7abe14ed0b55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:08.419299 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3cf702-16c9-45a4-ac98-7abe14ed0b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:08.419572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3cf702-16c9-45a4-ac98-7abe14ed0b55 admin admin] PATH_INFO: `/v3/projects/f78e58a1765d4dbe86a966e510f32cc2` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:08.420834 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3cf702-16c9-45a4-ac98-7abe14ed0b55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f78e58a1765d4dbe86a966e510f32cc2)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:08.421646 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fb806b-3450-4b0d-a6f2-8aae660a420a admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:08.422757 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3cf702-16c9-45a4-ac98-7abe14ed0b55 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:08.534201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1219/2438] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:08 2022] PUT /identity/v3/projects/74000cc7fc2845628610ca8b17ae395b/users/ab4b8cf9f04940428a8792a09f824378/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:08.545523 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23493b-6520-45f3-88a2-7d20f5d12752 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:08.545813 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23493b-6520-45f3-88a2-7d20f5d12752 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:08.546099 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea23493b-6520-45f3-88a2-7d20f5d12752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:08.578300 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2e3cf702-16c9-45a4-ac98-7abe14ed0b55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f78e58a1765d4dbe86a966e510f32cc2', 'request_id': 'req-2e3cf702-16c9-45a4-ac98-7abe14ed0b55'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:08.583785 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-ea23493b-6520-45f3-88a2-7d20f5d12752 None None] MFA Rules not processed for user `ab4b8cf9f04940428a8792a09f824378`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:08.585034 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-ea23493b-6520-45f3-88a2-7d20f5d12752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:08.624871 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:08.666720 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1220/2439] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:02:08.668507 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23493b-6520-45f3-88a2-7d20f5d12752 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:08.697649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2e3cf702-16c9-45a4-ac98-7abe14ed0b55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f78e58a1765d4dbe86a966e510f32cc2', 'request_id': 'req-2e3cf702-16c9-45a4-ac98-7abe14ed0b55'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:08.704704 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2e3cf702-16c9-45a4-ac98-7abe14ed0b55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f78e58a1765d4dbe86a966e510f32cc2', 'request_id': 'req-2e3cf702-16c9-45a4-ac98-7abe14ed0b55'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:08.719197 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2e3cf702-16c9-45a4-ac98-7abe14ed0b55 admin admin] The token cache is being invalidate because project f78e58a1765d4dbe86a966e510f32cc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:02:08.719479 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2e3cf702-16c9-45a4-ac98-7abe14ed0b55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:08.720158 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2e3cf702-16c9-45a4-ac98-7abe14ed0b55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:08.723099 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1220/2440] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:02:08 2022] DELETE /identity/v3/projects/f78e58a1765d4dbe86a966e510f32cc2 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:08.726115 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3cf702-16c9-45a4-ac98-7abe14ed0b55 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:08.729789 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-2e3cf702-16c9-45a4-ac98-7abe14ed0b55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:08.741613 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea23493b-6520-45f3-88a2-7d20f5d12752 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:08.743838 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3cf702-16c9-45a4-ac98-7abe14ed0b55 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:08.747103 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad37e6-97ce-48d6-86ae-1252a71227ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:08.747803 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad37e6-97ce-48d6-86ae-1252a71227ff admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:08.748152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad37e6-97ce-48d6-86ae-1252a71227ff admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:08.753045 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ad37e6-97ce-48d6-86ae-1252a71227ff admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:08.753045 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad37e6-97ce-48d6-86ae-1252a71227ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:08.753256 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7ad37-be5a-4e70-8ac5-ab6ab1b05378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:08.753256 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7ad37-be5a-4e70-8ac5-ab6ab1b05378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:08.753256 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7ad37-be5a-4e70-8ac5-ab6ab1b05378 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:08.753256 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a7ad37-be5a-4e70-8ac5-ab6ab1b05378 admin admin] PATH_INFO: `/v3/users/e40c68f50a0642bb9e846955fafc827d` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:08.753636 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a7ad37-be5a-4e70-8ac5-ab6ab1b05378 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e40c68f50a0642bb9e846955fafc827d)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:08.753726 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ad37e6-97ce-48d6-86ae-1252a71227ff admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:08.753817 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a7ad37-be5a-4e70-8ac5-ab6ab1b05378 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:08.915094 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1221/2441] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:02:08 2022] POST /identity/v3/projects => generated 372 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:08.916121 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad37e6-97ce-48d6-86ae-1252a71227ff admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:08.918713 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-49ad37e6-97ce-48d6-86ae-1252a71227ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:08.989600 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ad37e6-97ce-48d6-86ae-1252a71227ff admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:08.998845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=83,keystone:INSERT=5,keystone:DELETE=12 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:09.003968 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1365cd2-020b-4b89-a5e5-fa5be325a427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:09.005398 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1365cd2-020b-4b89-a5e5-fa5be325a427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:09.005398 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1365cd2-020b-4b89-a5e5-fa5be325a427 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:09.005398 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1365cd2-020b-4b89-a5e5-fa5be325a427 admin admin] PATH_INFO: `/v3/projects/406fced738034b25aa43e74c74f95af8` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:09.006451 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1365cd2-020b-4b89-a5e5-fa5be325a427 admin admin] RBAC: Authorizing `identity:delete_project(project_id=406fced738034b25aa43e74c74f95af8)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:09.011721 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1365cd2-020b-4b89-a5e5-fa5be325a427 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:09.017124 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-e4a7ad37-be5a-4e70-8ac5-ab6ab1b05378 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e40c68f50a0642bb9e846955fafc827d'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:09.025394 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-e4a7ad37-be5a-4e70-8ac5-ab6ab1b05378 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e40c68f50a0642bb9e846955fafc827d'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:09.026320 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-e4a7ad37-be5a-4e70-8ac5-ab6ab1b05378 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e40c68f50a0642bb9e846955fafc827d'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:09.080611 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-e4a7ad37-be5a-4e70-8ac5-ab6ab1b05378 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e40c68f50a0642bb9e846955fafc827d'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:09.094889 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-e4a7ad37-be5a-4e70-8ac5-ab6ab1b05378 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e40c68f50a0642bb9e846955fafc827d'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:09.099715 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c1365cd2-020b-4b89-a5e5-fa5be325a427 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '406fced738034b25aa43e74c74f95af8', 'request_id': 'req-c1365cd2-020b-4b89-a5e5-fa5be325a427'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:09.101112 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1221/2442] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:02:08 2022] DELETE /identity/v3/users/e40c68f50a0642bb9e846955fafc827d => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:02:09.103010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7ad37-be5a-4e70-8ac5-ab6ab1b05378 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:09.103702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-e4a7ad37-be5a-4e70-8ac5-ab6ab1b05378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:09.110308 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c1365cd2-020b-4b89-a5e5-fa5be325a427 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '406fced738034b25aa43e74c74f95af8', 'request_id': 'req-c1365cd2-020b-4b89-a5e5-fa5be325a427'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:09.124256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:09.172687 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a7ad37-be5a-4e70-8ac5-ab6ab1b05378 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:09.175874 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c1365cd2-020b-4b89-a5e5-fa5be325a427 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '406fced738034b25aa43e74c74f95af8', 'request_id': 'req-c1365cd2-020b-4b89-a5e5-fa5be325a427'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:09.179085 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d01d15-2f7b-4db0-bbb2-0c3377f3545a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:09.179720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d01d15-2f7b-4db0-bbb2-0c3377f3545a admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:09.180001 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d01d15-2f7b-4db0-bbb2-0c3377f3545a admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:09.180277 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d01d15-2f7b-4db0-bbb2-0c3377f3545a admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:09.180943 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d01d15-2f7b-4db0-bbb2-0c3377f3545a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:09.183650 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d01d15-2f7b-4db0-bbb2-0c3377f3545a admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:09.193033 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c1365cd2-020b-4b89-a5e5-fa5be325a427 admin admin] The token cache is being invalidate because project 406fced738034b25aa43e74c74f95af8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:02:09.193637 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c1365cd2-020b-4b89-a5e5-fa5be325a427 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:09.194532 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-c1365cd2-020b-4b89-a5e5-fa5be325a427 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:09.199439 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1222/2443] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:02:08 2022] DELETE /identity/v3/projects/406fced738034b25aa43e74c74f95af8 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:09.201257 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1365cd2-020b-4b89-a5e5-fa5be325a427 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:09.202165 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-c1365cd2-020b-4b89-a5e5-fa5be325a427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:09.267796 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1365cd2-020b-4b89-a5e5-fa5be325a427 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:09.273079 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac7ae1-f1d3-4443-8bd4-d3aa250a4a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:09.273757 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac7ae1-f1d3-4443-8bd4-d3aa250a4a6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:09.274287 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac7ae1-f1d3-4443-8bd4-d3aa250a4a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:09.274571 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac7ae1-f1d3-4443-8bd4-d3aa250a4a6e admin admin] PATH_INFO: `/v3/users/2ebd867695b541199f02cc754bb05bcc` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:09.275780 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac7ae1-f1d3-4443-8bd4-d3aa250a4a6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ebd867695b541199f02cc754bb05bcc)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:09.278605 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac7ae1-f1d3-4443-8bd4-d3aa250a4a6e admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:09.288459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1222/2444] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:02:09 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:09.292282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d01d15-2f7b-4db0-bbb2-0c3377f3545a admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:09.295532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-53d01d15-2f7b-4db0-bbb2-0c3377f3545a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:09.311389 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d01d15-2f7b-4db0-bbb2-0c3377f3545a admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:09.317513 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6e493-609b-4398-831b-4b2f292e4e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:09.318386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6e493-609b-4398-831b-4b2f292e4e06 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:09.318679 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6e493-609b-4398-831b-4b2f292e4e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:09.318958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6e493-609b-4398-831b-4b2f292e4e06 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:09.319787 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac6e493-609b-4398-831b-4b2f292e4e06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:09.321491 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac6e493-609b-4398-831b-4b2f292e4e06 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:09.336819 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1223/2445] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:09 2022] GET /identity/v3/roles => generated 1427 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:02:09.342061 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6e493-609b-4398-831b-4b2f292e4e06 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:09.352709 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6e493-609b-4398-831b-4b2f292e4e06 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:09.358753 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3163fb1-f664-4fac-b795-92ab22b96256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:09.365531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3163fb1-f664-4fac-b795-92ab22b96256 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:09.365531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3163fb1-f664-4fac-b795-92ab22b96256 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:09.365531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3163fb1-f664-4fac-b795-92ab22b96256 admin admin] PATH_INFO: `/v3/projects/b36e4e4fe12e4442b1e9b9d85d2b24ee/users/7c250ba3b9f245629428a220b4143c4f/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:09.380140 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3163fb1-f664-4fac-b795-92ab22b96256 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b36e4e4fe12e4442b1e9b9d85d2b24ee, user_id=7c250ba3b9f245629428a220b4143c4f, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:09.383115 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3163fb1-f664-4fac-b795-92ab22b96256 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:09.483558 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1224/2446] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:09 2022] PUT /identity/v3/projects/b36e4e4fe12e4442b1e9b9d85d2b24ee/users/7c250ba3b9f245629428a220b4143c4f/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:09.487995 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3163fb1-f664-4fac-b795-92ab22b96256 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:09.536179 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-1dac7ae1-f1d3-4443-8bd4-d3aa250a4a6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ebd867695b541199f02cc754bb05bcc'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:09.536952 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-1dac7ae1-f1d3-4443-8bd4-d3aa250a4a6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ebd867695b541199f02cc754bb05bcc'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:09.555196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3163fb1-f664-4fac-b795-92ab22b96256 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:09.561238 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce407628-1733-48cb-af27-328b5a53e493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:09.561874 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce407628-1733-48cb-af27-328b5a53e493 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:09.562160 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce407628-1733-48cb-af27-328b5a53e493 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:09.562460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce407628-1733-48cb-af27-328b5a53e493 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:09.563256 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce407628-1733-48cb-af27-328b5a53e493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:09.564925 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce407628-1733-48cb-af27-328b5a53e493 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:09.578587 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1225/2447] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:09 2022] GET /identity/v3/roles => generated 1427 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:02:09.583587 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce407628-1733-48cb-af27-328b5a53e493 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:09.595865 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce407628-1733-48cb-af27-328b5a53e493 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:09.602407 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2095a84-2600-4780-aadc-f669de32ed6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:09.603131 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2095a84-2600-4780-aadc-f669de32ed6d admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:09.603419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2095a84-2600-4780-aadc-f669de32ed6d admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:09.603698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2095a84-2600-4780-aadc-f669de32ed6d admin admin] PATH_INFO: `/v3/projects/b36e4e4fe12e4442b1e9b9d85d2b24ee/users/7c250ba3b9f245629428a220b4143c4f/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:09.605930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2095a84-2600-4780-aadc-f669de32ed6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b36e4e4fe12e4442b1e9b9d85d2b24ee, user_id=7c250ba3b9f245629428a220b4143c4f, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:09.608830 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2095a84-2600-4780-aadc-f669de32ed6d admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:09.622503 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1226/2448] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:09 2022] PUT /identity/v3/projects/b36e4e4fe12e4442b1e9b9d85d2b24ee/users/7c250ba3b9f245629428a220b4143c4f/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:02:09.623507 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2095a84-2600-4780-aadc-f669de32ed6d admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:09.625275 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-b2095a84-2600-4780-aadc-f669de32ed6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:09.668517 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-1dac7ae1-f1d3-4443-8bd4-d3aa250a4a6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ebd867695b541199f02cc754bb05bcc'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:09.676751 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-1dac7ae1-f1d3-4443-8bd4-d3aa250a4a6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ebd867695b541199f02cc754bb05bcc'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:09.695959 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-1dac7ae1-f1d3-4443-8bd4-d3aa250a4a6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ebd867695b541199f02cc754bb05bcc'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:09.703648 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1223/2449] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:02:09 2022] DELETE /identity/v3/users/2ebd867695b541199f02cc754bb05bcc => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:09.704187 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2095a84-2600-4780-aadc-f669de32ed6d admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:09.709295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386fcf4-4872-44a4-a2a7-8657ee2bdec3 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:09.709295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386fcf4-4872-44a4-a2a7-8657ee2bdec3 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:09.709295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386fcf4-4872-44a4-a2a7-8657ee2bdec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:09.753050 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-4386fcf4-4872-44a4-a2a7-8657ee2bdec3 None None] MFA Rules not processed for user `7c250ba3b9f245629428a220b4143c4f`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:09.755230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-4386fcf4-4872-44a4-a2a7-8657ee2bdec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:09.776623 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff635c-c387-4608-90d8-b4f9c6ff9ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:09.777268 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff635c-c387-4608-90d8-b4f9c6ff9ef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:09.777901 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff635c-c387-4608-90d8-b4f9c6ff9ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:09.778733 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff635c-c387-4608-90d8-b4f9c6ff9ef9 admin admin] PATH_INFO: `/v3/projects/1f1d31337aaa47d2854acd58e35bc94e` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:09.781437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff635c-c387-4608-90d8-b4f9c6ff9ef9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f1d31337aaa47d2854acd58e35bc94e)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:09.784144 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff635c-c387-4608-90d8-b4f9c6ff9ef9 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:09.800568 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:09.845416 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1224/2450] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:09 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:09.857320 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386fcf4-4872-44a4-a2a7-8657ee2bdec3 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:09.859415 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-4386fcf4-4872-44a4-a2a7-8657ee2bdec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:09.917402 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:09.921754 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-f9ff635c-c387-4608-90d8-b4f9c6ff9ef9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f1d31337aaa47d2854acd58e35bc94e', 'request_id': 'req-f9ff635c-c387-4608-90d8-b4f9c6ff9ef9'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:09.938664 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386fcf4-4872-44a4-a2a7-8657ee2bdec3 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:09.997672 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45328a-76e7-4268-a9eb-9b272e24d78a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:09.998386 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45328a-76e7-4268-a9eb-9b272e24d78a service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:09.999469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45328a-76e7-4268-a9eb-9b272e24d78a service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:09.999469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec45328a-76e7-4268-a9eb-9b272e24d78a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:10.014697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-f9ff635c-c387-4608-90d8-b4f9c6ff9ef9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f1d31337aaa47d2854acd58e35bc94e', 'request_id': 'req-f9ff635c-c387-4608-90d8-b4f9c6ff9ef9'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:10.022307 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-f9ff635c-c387-4608-90d8-b4f9c6ff9ef9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f1d31337aaa47d2854acd58e35bc94e', 'request_id': 'req-f9ff635c-c387-4608-90d8-b4f9c6ff9ef9'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:10.037258 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-f9ff635c-c387-4608-90d8-b4f9c6ff9ef9 admin admin] The token cache is being invalidate because project 1f1d31337aaa47d2854acd58e35bc94e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:02:10.037543 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-f9ff635c-c387-4608-90d8-b4f9c6ff9ef9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:10.038415 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-f9ff635c-c387-4608-90d8-b4f9c6ff9ef9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:10.040401 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1227/2451] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:02:09 2022] DELETE /identity/v3/projects/1f1d31337aaa47d2854acd58e35bc94e => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:10.045902 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a6a37-1c0a-4286-ba6d-546672c0dd4f None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:10.046212 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a6a37-1c0a-4286-ba6d-546672c0dd4f None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:10.046455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4a6a37-1c0a-4286-ba6d-546672c0dd4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:10.069460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-0c4a6a37-1c0a-4286-ba6d-546672c0dd4f None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:10.070955 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-0c4a6a37-1c0a-4286-ba6d-546672c0dd4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:10.080970 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec45328a-76e7-4268-a9eb-9b272e24d78a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:10.082439 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec45328a-76e7-4268-a9eb-9b272e24d78a service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:10.094461 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1225/2452] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:09 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:02:10.095681 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45328a-76e7-4268-a9eb-9b272e24d78a service cinder] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:10.097161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-ec45328a-76e7-4268-a9eb-9b272e24d78a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:10.109824 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec45328a-76e7-4268-a9eb-9b272e24d78a service cinder] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:10.114961 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96acc553-b036-4b0f-ad74-0646a4ec2ec3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:10.115833 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acc553-b036-4b0f-ad74-0646a4ec2ec3 service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:10.115980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acc553-b036-4b0f-ad74-0646a4ec2ec3 service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:10.118319 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96acc553-b036-4b0f-ad74-0646a4ec2ec3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:10.119013 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-96acc553-b036-4b0f-ad74-0646a4ec2ec3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:10.163343 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1228/2453] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:10.173416 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799250cd-fb90-4c85-889d-8fc3a7bdcf1c None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:10.173703 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799250cd-fb90-4c85-889d-8fc3a7bdcf1c None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:10.174003 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799250cd-fb90-4c85-889d-8fc3a7bdcf1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:10.195409 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-799250cd-fb90-4c85-889d-8fc3a7bdcf1c None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:10.196771 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-799250cd-fb90-4c85-889d-8fc3a7bdcf1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:10.205211 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96acc553-b036-4b0f-ad74-0646a4ec2ec3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:10.206723 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96acc553-b036-4b0f-ad74-0646a4ec2ec3 service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:10.213978 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1229/2454] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:02:10.215661 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799250cd-fb90-4c85-889d-8fc3a7bdcf1c None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:10.217590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1226/2455] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:10 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:02:10.229632 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799250cd-fb90-4c85-889d-8fc3a7bdcf1c None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:10.242786 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96acc553-b036-4b0f-ad74-0646a4ec2ec3 service cinder] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:10.242786 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-96acc553-b036-4b0f-ad74-0646a4ec2ec3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:10.243247 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54746d98-3284-4dfa-baa2-33af4e099be2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:10.247668 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54746d98-3284-4dfa-baa2-33af4e099be2 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:10.253243 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54746d98-3284-4dfa-baa2-33af4e099be2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:10.253243 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54746d98-3284-4dfa-baa2-33af4e099be2 admin admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:10.254214 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54746d98-3284-4dfa-baa2-33af4e099be2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:02:10.254733 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54746d98-3284-4dfa-baa2-33af4e099be2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:10.256457 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96acc553-b036-4b0f-ad74-0646a4ec2ec3 service cinder] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:10.259010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54746d98-3284-4dfa-baa2-33af4e099be2 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:10.269359 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa53b53-f176-4ea4-a392-8767a75aca3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:10.270017 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa53b53-f176-4ea4-a392-8767a75aca3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:10.270809 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa53b53-f176-4ea4-a392-8767a75aca3e admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:10.271199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa53b53-f176-4ea4-a392-8767a75aca3e admin admin] PATH_INFO: `/v3/projects/eabf9882be3f46aeb1835813241953eb` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:10.272651 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa53b53-f176-4ea4-a392-8767a75aca3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=eabf9882be3f46aeb1835813241953eb)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:10.274886 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa53b53-f176-4ea4-a392-8767a75aca3e admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:10.283610 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1230/2456] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 11:02:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:02:10.284635 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54746d98-3284-4dfa-baa2-33af4e099be2 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:10.301697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54746d98-3284-4dfa-baa2-33af4e099be2 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:10.304055 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283272a-bd2b-4810-a2d6-e711e3f80edd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:10.304940 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283272a-bd2b-4810-a2d6-e711e3f80edd admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:10.305455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283272a-bd2b-4810-a2d6-e711e3f80edd admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:10.305914 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283272a-bd2b-4810-a2d6-e711e3f80edd admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:10.306994 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5283272a-bd2b-4810-a2d6-e711e3f80edd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:10.311276 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5283272a-bd2b-4810-a2d6-e711e3f80edd admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:10.420817 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1231/2457] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:02:10 2022] POST /identity/v3/projects => generated 382 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:10.421655 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283272a-bd2b-4810-a2d6-e711e3f80edd admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:10.423299 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-5283272a-bd2b-4810-a2d6-e711e3f80edd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:10.439112 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-1fa53b53-f176-4ea4-a392-8767a75aca3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eabf9882be3f46aeb1835813241953eb', 'request_id': 'req-1fa53b53-f176-4ea4-a392-8767a75aca3e'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:10.449404 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-1fa53b53-f176-4ea4-a392-8767a75aca3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eabf9882be3f46aeb1835813241953eb', 'request_id': 'req-1fa53b53-f176-4ea4-a392-8767a75aca3e'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:10.482086 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283272a-bd2b-4810-a2d6-e711e3f80edd admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:10.485862 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae987ec4-35e9-4d9c-9230-40698b9b632d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:10.486482 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae987ec4-35e9-4d9c-9230-40698b9b632d service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:10.486766 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae987ec4-35e9-4d9c-9230-40698b9b632d service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:10.487136 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae987ec4-35e9-4d9c-9230-40698b9b632d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:10.489008 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-ae987ec4-35e9-4d9c-9230-40698b9b632d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:10.521253 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-1fa53b53-f176-4ea4-a392-8767a75aca3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eabf9882be3f46aeb1835813241953eb', 'request_id': 'req-1fa53b53-f176-4ea4-a392-8767a75aca3e'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:10.536859 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-1fa53b53-f176-4ea4-a392-8767a75aca3e admin admin] The token cache is being invalidate because project eabf9882be3f46aeb1835813241953eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:02:10.537249 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-1fa53b53-f176-4ea4-a392-8767a75aca3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:10.538034 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-1fa53b53-f176-4ea4-a392-8767a75aca3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:10.540116 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1227/2458] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:02:10 2022] DELETE /identity/v3/projects/eabf9882be3f46aeb1835813241953eb => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:02:10.540801 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa53b53-f176-4ea4-a392-8767a75aca3e admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:10.544163 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-1fa53b53-f176-4ea4-a392-8767a75aca3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:10.561329 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae987ec4-35e9-4d9c-9230-40698b9b632d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:10.562928 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae987ec4-35e9-4d9c-9230-40698b9b632d service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:10.572395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1232/2459] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:10 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:02:10.578891 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75b6a5-7875-43cd-b96f-c6f5d36373f9 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:10.578891 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75b6a5-7875-43cd-b96f-c6f5d36373f9 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:10.578891 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75b6a5-7875-43cd-b96f-c6f5d36373f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:10.595165 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-df75b6a5-7875-43cd-b96f-c6f5d36373f9 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:10.596479 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-df75b6a5-7875-43cd-b96f-c6f5d36373f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:10.619129 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa53b53-f176-4ea4-a392-8767a75aca3e admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:10.665985 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1233/2460] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:10.677550 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add83b89-d6e7-41af-934e-3b89b9643604 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:10.678732 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add83b89-d6e7-41af-934e-3b89b9643604 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:10.678732 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add83b89-d6e7-41af-934e-3b89b9643604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:10.685572 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8218dade-6c69-4084-b7a8-1755f0bab5d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:10.687911 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218dade-6c69-4084-b7a8-1755f0bab5d4 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:10.687911 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218dade-6c69-4084-b7a8-1755f0bab5d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:10.687911 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218dade-6c69-4084-b7a8-1755f0bab5d4 admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:10.687911 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8218dade-6c69-4084-b7a8-1755f0bab5d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:10.689673 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8218dade-6c69-4084-b7a8-1755f0bab5d4 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:10.702850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-add83b89-d6e7-41af-934e-3b89b9643604 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:10.704181 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-add83b89-d6e7-41af-934e-3b89b9643604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:10.712358 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1234/2461] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:02:10.717118 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add83b89-d6e7-41af-934e-3b89b9643604 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:10.726709 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add83b89-d6e7-41af-934e-3b89b9643604 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:10.732429 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fb50e-0db9-4b6c-81b0-b1c2f0d65ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:10.733170 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fb50e-0db9-4b6c-81b0-b1c2f0d65ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:10.733450 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fb50e-0db9-4b6c-81b0-b1c2f0d65ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:10.738427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880fb50e-0db9-4b6c-81b0-b1c2f0d65ee7 admin admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:10.738427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880fb50e-0db9-4b6c-81b0-b1c2f0d65ee7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:02:10.738427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880fb50e-0db9-4b6c-81b0-b1c2f0d65ee7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:10.738427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880fb50e-0db9-4b6c-81b0-b1c2f0d65ee7 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:10.748535 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1235/2462] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 11:02:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 11:02:10.755818 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fb50e-0db9-4b6c-81b0-b1c2f0d65ee7 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:10.766173 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880fb50e-0db9-4b6c-81b0-b1c2f0d65ee7 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:10.771177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8087627-75c8-4bea-a62b-75081dfcdcbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:10.771782 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8087627-75c8-4bea-a62b-75081dfcdcbb admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:10.772058 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8087627-75c8-4bea-a62b-75081dfcdcbb admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:10.772329 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8087627-75c8-4bea-a62b-75081dfcdcbb admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:10.773044 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8087627-75c8-4bea-a62b-75081dfcdcbb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:10.775249 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8087627-75c8-4bea-a62b-75081dfcdcbb admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:10.814971 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1228/2463] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:02:10 2022] POST /identity/v3/users => generated 292 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:10.816151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8218dade-6c69-4084-b7a8-1755f0bab5d4 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:10.827684 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8218dade-6c69-4084-b7a8-1755f0bab5d4 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:10.832584 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b0ac8-a232-40b2-b7c2-36d625cbe712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:10.836466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b0ac8-a232-40b2-b7c2-36d625cbe712 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:10.836466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b0ac8-a232-40b2-b7c2-36d625cbe712 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:10.836466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b0ac8-a232-40b2-b7c2-36d625cbe712 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:10.836466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19b0ac8-a232-40b2-b7c2-36d625cbe712 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:10.836466 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19b0ac8-a232-40b2-b7c2-36d625cbe712 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:10.847579 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1229/2464] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:10 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:10.851633 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b0ac8-a232-40b2-b7c2-36d625cbe712 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:10.861884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19b0ac8-a232-40b2-b7c2-36d625cbe712 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:10.866520 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf53c9-8412-4115-a99d-3ae888e04796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:10.867157 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf53c9-8412-4115-a99d-3ae888e04796 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:10.867438 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf53c9-8412-4115-a99d-3ae888e04796 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:10.867707 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daaf53c9-8412-4115-a99d-3ae888e04796 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:10.868401 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf53c9-8412-4115-a99d-3ae888e04796 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:10.870036 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daaf53c9-8412-4115-a99d-3ae888e04796 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:10.881201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1230/2465] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:10 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:02:10.885349 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf53c9-8412-4115-a99d-3ae888e04796 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:10.896242 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daaf53c9-8412-4115-a99d-3ae888e04796 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:10.902885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4d69ea-92b7-47ca-b5cf-e1ad1fef5202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:10.903380 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1236/2466] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:02:10 2022] POST /identity/v3/projects => generated 400 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:02:10.903659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d69ea-92b7-47ca-b5cf-e1ad1fef5202 admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:10.903947 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d69ea-92b7-47ca-b5cf-e1ad1fef5202 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:10.904250 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4d69ea-92b7-47ca-b5cf-e1ad1fef5202 admin admin] PATH_INFO: `/v3/projects/b734c17e6b404d2dbce69eb4b4faf3da/users/474f367bbcf64cc48d0399ccd67df833/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:10.907643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8087627-75c8-4bea-a62b-75081dfcdcbb admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:10.917545 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:10.923089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d69ea-92b7-47ca-b5cf-e1ad1fef5202 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b734c17e6b404d2dbce69eb4b4faf3da, user_id=474f367bbcf64cc48d0399ccd67df833, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:10.925772 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4d69ea-92b7-47ca-b5cf-e1ad1fef5202 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:10.975376 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:10.989072 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8087627-75c8-4bea-a62b-75081dfcdcbb admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:10.993768 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc192329-4ba1-4dbb-aeac-21af47aa8e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:10.994410 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc192329-4ba1-4dbb-aeac-21af47aa8e76 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:10.994691 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc192329-4ba1-4dbb-aeac-21af47aa8e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:10.994963 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc192329-4ba1-4dbb-aeac-21af47aa8e76 admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:10.995791 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc192329-4ba1-4dbb-aeac-21af47aa8e76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:10.998256 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc192329-4ba1-4dbb-aeac-21af47aa8e76 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:11.010164 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1231/2467] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:10 2022] PUT /identity/v3/projects/b734c17e6b404d2dbce69eb4b4faf3da/users/474f367bbcf64cc48d0399ccd67df833/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:02:11.030777 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33c2ac-03fc-4e2f-bee8-763d2b848626 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:11.034201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33c2ac-03fc-4e2f-bee8-763d2b848626 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:11.034201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e33c2ac-03fc-4e2f-bee8-763d2b848626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:11.084666 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-6e33c2ac-03fc-4e2f-bee8-763d2b848626 None None] MFA Rules not processed for user `474f367bbcf64cc48d0399ccd67df833`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:11.085864 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-6e33c2ac-03fc-4e2f-bee8-763d2b848626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:11.109122 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1237/2468] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:02:10 2022] POST /identity/v3/users => generated 301 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:02:11.113463 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc192329-4ba1-4dbb-aeac-21af47aa8e76 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:11.132567 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc192329-4ba1-4dbb-aeac-21af47aa8e76 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:11.159149 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1232/2469] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:11 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:11.163597 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33c2ac-03fc-4e2f-bee8-763d2b848626 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:11.198904 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32ab55-edf1-48ba-b9b6-7f670c1732ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:11.200239 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32ab55-edf1-48ba-b9b6-7f670c1732ac admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:11.200756 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32ab55-edf1-48ba-b9b6-7f670c1732ac admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:11.201303 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f32ab55-edf1-48ba-b9b6-7f670c1732ac admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:11.202730 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f32ab55-edf1-48ba-b9b6-7f670c1732ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:11.203883 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f32ab55-edf1-48ba-b9b6-7f670c1732ac admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:11.219132 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1238/2470] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:11 2022] GET /identity/v3/roles => generated 1427 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:02:11.220125 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32ab55-edf1-48ba-b9b6-7f670c1732ac admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:11.232880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f32ab55-edf1-48ba-b9b6-7f670c1732ac admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:11.238648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025cce42-3f68-4935-b8a1-7d98c44391a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:11.238963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e33c2ac-03fc-4e2f-bee8-763d2b848626 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:11.239418 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cce42-3f68-4935-b8a1-7d98c44391a0 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:11.239767 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cce42-3f68-4935-b8a1-7d98c44391a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:11.240108 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cce42-3f68-4935-b8a1-7d98c44391a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:11.241012 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025cce42-3f68-4935-b8a1-7d98c44391a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:11.243252 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025cce42-3f68-4935-b8a1-7d98c44391a0 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:11.248035 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd7748-e47c-4307-955c-28c1b2c4bb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:11.248568 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd7748-e47c-4307-955c-28c1b2c4bb1a admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:11.248907 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd7748-e47c-4307-955c-28c1b2c4bb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:11.249253 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd7748-e47c-4307-955c-28c1b2c4bb1a admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:11.250207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd7748-e47c-4307-955c-28c1b2c4bb1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:11.252239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd7748-e47c-4307-955c-28c1b2c4bb1a admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:11.255820 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1239/2471] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:11 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:11.260461 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025cce42-3f68-4935-b8a1-7d98c44391a0 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:11.272673 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025cce42-3f68-4935-b8a1-7d98c44391a0 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:11.278953 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8b335-c195-424c-a4c4-f828d41ff16c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:11.279776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8b335-c195-424c-a4c4-f828d41ff16c admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:11.280156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8b335-c195-424c-a4c4-f828d41ff16c admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:11.280518 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8b335-c195-424c-a4c4-f828d41ff16c admin admin] PATH_INFO: `/v3/projects/a7b6e1a86a7e4876a222b007067c94ff/users/a039ffa0adf04575b5493c622735e5b7/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:11.299844 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8b335-c195-424c-a4c4-f828d41ff16c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7b6e1a86a7e4876a222b007067c94ff, user_id=a039ffa0adf04575b5493c622735e5b7, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:11.302736 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8b335-c195-424c-a4c4-f828d41ff16c admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:11.357405 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1233/2472] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:02:11 2022] POST /identity/v3/projects => generated 380 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:02:11.362345 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd7748-e47c-4307-955c-28c1b2c4bb1a admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:11.394682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1240/2473] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:11 2022] PUT /identity/v3/projects/a7b6e1a86a7e4876a222b007067c94ff/users/a039ffa0adf04575b5493c622735e5b7/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:11.405969 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0cfff-ef8e-4cf0-8d42-dde6377651c4 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:11.406254 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0cfff-ef8e-4cf0-8d42-dde6377651c4 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:11.406603 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0cfff-ef8e-4cf0-8d42-dde6377651c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:11.429384 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd7748-e47c-4307-955c-28c1b2c4bb1a admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:11.436729 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-d2c0cfff-ef8e-4cf0-8d42-dde6377651c4 None None] MFA Rules not processed for user `a039ffa0adf04575b5493c622735e5b7`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:11.446002 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-d2c0cfff-ef8e-4cf0-8d42-dde6377651c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:11.493851 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7e67a-2680-49f6-b815-8e755406948c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:11.494479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7e67a-2680-49f6-b815-8e755406948c admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:11.494762 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7e67a-2680-49f6-b815-8e755406948c admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:11.495037 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a7e67a-2680-49f6-b815-8e755406948c admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:11.496930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a7e67a-2680-49f6-b815-8e755406948c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:11.498442 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a7e67a-2680-49f6-b815-8e755406948c admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:11.518010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1241/2474] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:11 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:02:11.522799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0cfff-ef8e-4cf0-8d42-dde6377651c4 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:11.533730 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0cfff-ef8e-4cf0-8d42-dde6377651c4 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:11.538182 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3421167f-685b-48c8-9444-fa5ac7556e11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:11.538799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421167f-685b-48c8-9444-fa5ac7556e11 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:11.539181 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421167f-685b-48c8-9444-fa5ac7556e11 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:11.539537 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3421167f-685b-48c8-9444-fa5ac7556e11 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:11.540191 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3421167f-685b-48c8-9444-fa5ac7556e11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:11.543478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:11.543667 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3421167f-685b-48c8-9444-fa5ac7556e11 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:11.630816 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1242/2475] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:02:11 2022] POST /identity/v3/projects => generated 400 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:02:11.633804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1234/2476] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:02:11 2022] POST /identity/v3/users => generated 297 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:02:11.634565 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3421167f-685b-48c8-9444-fa5ac7556e11 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:11.640361 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7e67a-2680-49f6-b815-8e755406948c admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:11.651966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a7e67a-2680-49f6-b815-8e755406948c admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:11.699575 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3421167f-685b-48c8-9444-fa5ac7556e11 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:11.704498 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d8c804-ec74-40a8-9f9c-8f25a31583ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:11.705113 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8c804-ec74-40a8-9f9c-8f25a31583ba admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:11.705396 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8c804-ec74-40a8-9f9c-8f25a31583ba admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:11.705670 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d8c804-ec74-40a8-9f9c-8f25a31583ba admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:11.706478 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d8c804-ec74-40a8-9f9c-8f25a31583ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:11.708165 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d8c804-ec74-40a8-9f9c-8f25a31583ba admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:11.714737 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502e52e-8ef3-4ffe-9923-60772f57477f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:11.715385 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502e52e-8ef3-4ffe-9923-60772f57477f admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:11.715669 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502e52e-8ef3-4ffe-9923-60772f57477f admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:11.715943 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3502e52e-8ef3-4ffe-9923-60772f57477f admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:11.716612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3502e52e-8ef3-4ffe-9923-60772f57477f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:11.718536 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3502e52e-8ef3-4ffe-9923-60772f57477f admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:11.718776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1243/2477] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:11 2022] GET /identity/v3/roles => generated 1427 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:11.723157 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d8c804-ec74-40a8-9f9c-8f25a31583ba admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:11.734102 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d8c804-ec74-40a8-9f9c-8f25a31583ba admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:11.738880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58d90e-5ed8-473a-b38b-b7d7d2a33280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:11.739584 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58d90e-5ed8-473a-b38b-b7d7d2a33280 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:11.739874 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58d90e-5ed8-473a-b38b-b7d7d2a33280 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:11.740150 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf58d90e-5ed8-473a-b38b-b7d7d2a33280 admin admin] PATH_INFO: `/v3/projects/5a320e265e2e42049bb67ff8e7c4096a/users/d71a54f8e0bc409a8a07d3da11cdbc70/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:11.756046 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58d90e-5ed8-473a-b38b-b7d7d2a33280 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a320e265e2e42049bb67ff8e7c4096a, user_id=d71a54f8e0bc409a8a07d3da11cdbc70, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:11.758770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf58d90e-5ed8-473a-b38b-b7d7d2a33280 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:11.801020 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1235/2478] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:02:11 2022] POST /identity/v3/users => generated 307 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:02:11.805334 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502e52e-8ef3-4ffe-9923-60772f57477f admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:11.816149 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3502e52e-8ef3-4ffe-9923-60772f57477f admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:11.821042 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febeb721-9f41-4c91-a4af-9e6396517a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:11.821659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febeb721-9f41-4c91-a4af-9e6396517a41 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:11.821939 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febeb721-9f41-4c91-a4af-9e6396517a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:11.822218 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febeb721-9f41-4c91-a4af-9e6396517a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:11.822907 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febeb721-9f41-4c91-a4af-9e6396517a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:11.824772 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febeb721-9f41-4c91-a4af-9e6396517a41 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:11.835745 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1236/2479] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:11 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:02:11.840465 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febeb721-9f41-4c91-a4af-9e6396517a41 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:11.851766 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1244/2480] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:11 2022] PUT /identity/v3/projects/5a320e265e2e42049bb67ff8e7c4096a/users/d71a54f8e0bc409a8a07d3da11cdbc70/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:02:11.852084 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febeb721-9f41-4c91-a4af-9e6396517a41 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:11.866643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58d90e-5ed8-473a-b38b-b7d7d2a33280 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:11.923355 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25fa951-4f77-438e-9bc6-73ee16412d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:11.924203 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25fa951-4f77-438e-9bc6-73ee16412d1c admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:11.924331 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25fa951-4f77-438e-9bc6-73ee16412d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:11.924642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25fa951-4f77-438e-9bc6-73ee16412d1c admin admin] PATH_INFO: `/v3/projects/f2c1c2f4f3184068928d3155115b37ae/users/31db839d54544ddd990cfc74f53b436a/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:11.938670 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf58d90e-5ed8-473a-b38b-b7d7d2a33280 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:11.941438 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25fa951-4f77-438e-9bc6-73ee16412d1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2c1c2f4f3184068928d3155115b37ae, user_id=31db839d54544ddd990cfc74f53b436a, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:11.943458 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0aa2b9-2896-4833-b960-ec6a68a76fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:11.944069 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0aa2b9-2896-4833-b960-ec6a68a76fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:11.944196 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25fa951-4f77-438e-9bc6-73ee16412d1c admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:11.944350 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0aa2b9-2896-4833-b960-ec6a68a76fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:11.944625 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0aa2b9-2896-4833-b960-ec6a68a76fd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:11.945312 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0aa2b9-2896-4833-b960-ec6a68a76fd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:11.946895 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0aa2b9-2896-4833-b960-ec6a68a76fd1 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:11.957100 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1245/2481] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:11 2022] GET /identity/v3/roles => generated 1427 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:02:11.961388 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0aa2b9-2896-4833-b960-ec6a68a76fd1 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:11.972499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0aa2b9-2896-4833-b960-ec6a68a76fd1 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:11.977538 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759dc555-c6a8-462e-93cb-78cc969d011e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:11.978300 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759dc555-c6a8-462e-93cb-78cc969d011e admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:11.978579 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759dc555-c6a8-462e-93cb-78cc969d011e admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:11.978851 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759dc555-c6a8-462e-93cb-78cc969d011e admin admin] PATH_INFO: `/v3/projects/5a320e265e2e42049bb67ff8e7c4096a/users/d71a54f8e0bc409a8a07d3da11cdbc70/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:11.981383 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759dc555-c6a8-462e-93cb-78cc969d011e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a320e265e2e42049bb67ff8e7c4096a, user_id=d71a54f8e0bc409a8a07d3da11cdbc70, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:11.984219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759dc555-c6a8-462e-93cb-78cc969d011e admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:11.994927 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1246/2482] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:11 2022] PUT /identity/v3/projects/5a320e265e2e42049bb67ff8e7c4096a/users/d71a54f8e0bc409a8a07d3da11cdbc70/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:02:12.005803 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61103e9d-ad0e-43e0-a52d-9829d32f3e66 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:12.006086 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61103e9d-ad0e-43e0-a52d-9829d32f3e66 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:12.006364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61103e9d-ad0e-43e0-a52d-9829d32f3e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:12.036985 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-61103e9d-ad0e-43e0-a52d-9829d32f3e66 None None] MFA Rules not processed for user `d71a54f8e0bc409a8a07d3da11cdbc70`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:12.038227 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-61103e9d-ad0e-43e0-a52d-9829d32f3e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:12.063546 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1237/2483] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:11 2022] PUT /identity/v3/projects/f2c1c2f4f3184068928d3155115b37ae/users/31db839d54544ddd990cfc74f53b436a/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:12.067588 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25fa951-4f77-438e-9bc6-73ee16412d1c admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:12.110548 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1247/2484] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:12 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:02:12.123732 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61103e9d-ad0e-43e0-a52d-9829d32f3e66 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:12.135980 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-61103e9d-ad0e-43e0-a52d-9829d32f3e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:12.137410 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25fa951-4f77-438e-9bc6-73ee16412d1c admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:12.142385 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3797e35-dee9-45a3-b746-dd000f2a2d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:12.143012 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3797e35-dee9-45a3-b746-dd000f2a2d5c admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:12.143331 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3797e35-dee9-45a3-b746-dd000f2a2d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:12.143610 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3797e35-dee9-45a3-b746-dd000f2a2d5c admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:12.144457 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3797e35-dee9-45a3-b746-dd000f2a2d5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:12.146057 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3797e35-dee9-45a3-b746-dd000f2a2d5c admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:12.158695 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1238/2485] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:12 2022] GET /identity/v3/roles => generated 1427 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:02:12.162740 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3797e35-dee9-45a3-b746-dd000f2a2d5c admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:12.181185 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3797e35-dee9-45a3-b746-dd000f2a2d5c admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:12.192712 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e812dd-b2e0-449e-953b-3eb224836628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:12.193573 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e812dd-b2e0-449e-953b-3eb224836628 admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:12.193863 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e812dd-b2e0-449e-953b-3eb224836628 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:12.194207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e812dd-b2e0-449e-953b-3eb224836628 admin admin] PATH_INFO: `/v3/projects/f2c1c2f4f3184068928d3155115b37ae/users/31db839d54544ddd990cfc74f53b436a/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:12.196663 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e812dd-b2e0-449e-953b-3eb224836628 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2c1c2f4f3184068928d3155115b37ae, user_id=31db839d54544ddd990cfc74f53b436a, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:12.199528 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61103e9d-ad0e-43e0-a52d-9829d32f3e66 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:12.200247 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e812dd-b2e0-449e-953b-3eb224836628 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:12.212720 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1239/2486] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:12 2022] PUT /identity/v3/projects/f2c1c2f4f3184068928d3155115b37ae/users/31db839d54544ddd990cfc74f53b436a/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:02:12.225234 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf6782-f9bb-473d-8551-4e7ddf198698 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:12.225527 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf6782-f9bb-473d-8551-4e7ddf198698 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:12.225802 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf6782-f9bb-473d-8551-4e7ddf198698 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:12.237489 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:12.256052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-87bf6782-f9bb-473d-8551-4e7ddf198698 None None] MFA Rules not processed for user `31db839d54544ddd990cfc74f53b436a`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:12.257037 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-87bf6782-f9bb-473d-8551-4e7ddf198698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:12.259007 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88789c3b-8e91-4a6c-9bbe-d61ade0b7f1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:12.259932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88789c3b-8e91-4a6c-9bbe-d61ade0b7f1d service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:12.260294 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88789c3b-8e91-4a6c-9bbe-d61ade0b7f1d service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:12.260670 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88789c3b-8e91-4a6c-9bbe-d61ade0b7f1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:12.332582 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88789c3b-8e91-4a6c-9bbe-d61ade0b7f1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:12.334080 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88789c3b-8e91-4a6c-9bbe-d61ade0b7f1d service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:12.337374 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1240/2487] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:12 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:02:12.347122 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1248/2488] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:12 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:02:12.429975 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf6782-f9bb-473d-8551-4e7ddf198698 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:12.495973 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bf6782-f9bb-473d-8551-4e7ddf198698 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:12.499875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a902d1-49b5-4793-b0e5-2be404376742 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:12.500501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a902d1-49b5-4793-b0e5-2be404376742 service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:12.500780 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a902d1-49b5-4793-b0e5-2be404376742 service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:12.501155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a902d1-49b5-4793-b0e5-2be404376742 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:12.511955 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a902d1-49b5-4793-b0e5-2be404376742 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:12.513477 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a902d1-49b5-4793-b0e5-2be404376742 service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:12.524941 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1241/2489] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:12 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:02:12.609545 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:12.701540 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:13.002531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88789c3b-8e91-4a6c-9bbe-d61ade0b7f1d service cinder] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:13.004165 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-88789c3b-8e91-4a6c-9bbe-d61ade0b7f1d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:13.017386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88789c3b-8e91-4a6c-9bbe-d61ade0b7f1d service cinder] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:13.022032 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bea809-dee8-47ef-8a9f-e5797ce8280f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:13.022655 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bea809-dee8-47ef-8a9f-e5797ce8280f service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:13.023204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bea809-dee8-47ef-8a9f-e5797ce8280f service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:13.023592 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bea809-dee8-47ef-8a9f-e5797ce8280f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:13.088008 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bea809-dee8-47ef-8a9f-e5797ce8280f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:13.088658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:13.089739 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bea809-dee8-47ef-8a9f-e5797ce8280f service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:13.099578 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1249/2490] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:13 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:02:13.246728 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:13.795376 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:14.085979 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:14.245186 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a902d1-49b5-4793-b0e5-2be404376742 service cinder] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:14.316081 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a902d1-49b5-4793-b0e5-2be404376742 service cinder] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:14.321005 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c4a69-afd8-4878-bf76-d016cca2ec9d tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd71a54f8e0bc409a8a07d3da11cdbc70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a320e265e2e42049bb67ff8e7c4096a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:14.321624 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c4a69-afd8-4878-bf76-d016cca2ec9d tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:14.324476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c4a69-afd8-4878-bf76-d016cca2ec9d tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:14.324476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113c4a69-afd8-4878-bf76-d016cca2ec9d tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:14.324476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113c4a69-afd8-4878-bf76-d016cca2ec9d tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:02:14.324476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113c4a69-afd8-4878-bf76-d016cca2ec9d tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:14.325186 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113c4a69-afd8-4878-bf76-d016cca2ec9d tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:14.337498 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1242/2491] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 11:02:14 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:02:14.342917 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bea809-dee8-47ef-8a9f-e5797ce8280f service cinder] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:14.357321 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bea809-dee8-47ef-8a9f-e5797ce8280f service cinder] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:14.363309 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e47a5f-1428-4575-8387-97c35abaff28 tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd71a54f8e0bc409a8a07d3da11cdbc70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a320e265e2e42049bb67ff8e7c4096a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:14.363933 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e47a5f-1428-4575-8387-97c35abaff28 tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:14.364280 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e47a5f-1428-4575-8387-97c35abaff28 tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:14.364630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e47a5f-1428-4575-8387-97c35abaff28 tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:14.365318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e47a5f-1428-4575-8387-97c35abaff28 tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:14.367266 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e47a5f-1428-4575-8387-97c35abaff28 tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:14.527695 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1250/2492] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:02:14 2022] POST /identity/v3/projects => generated 354 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:14.531948 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c4a69-afd8-4878-bf76-d016cca2ec9d tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:14.542111 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113c4a69-afd8-4878-bf76-d016cca2ec9d tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:14.604150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097dda45-626b-42a8-9dbe-e509a07227ce tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd71a54f8e0bc409a8a07d3da11cdbc70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a320e265e2e42049bb67ff8e7c4096a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:14.605325 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097dda45-626b-42a8-9dbe-e509a07227ce tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:14.605626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097dda45-626b-42a8-9dbe-e509a07227ce tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:14.605960 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097dda45-626b-42a8-9dbe-e509a07227ce tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:14.606777 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097dda45-626b-42a8-9dbe-e509a07227ce tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:02:14.607112 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097dda45-626b-42a8-9dbe-e509a07227ce tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:14.608848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097dda45-626b-42a8-9dbe-e509a07227ce tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:14.621661 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1243/2493] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 11:02:14 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:02:14.665007 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e47a5f-1428-4575-8387-97c35abaff28 tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:14.676807 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e47a5f-1428-4575-8387-97c35abaff28 tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:14.681909 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b355e4-e5fa-4113-8e0f-d60dd317cc29 tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd71a54f8e0bc409a8a07d3da11cdbc70', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a320e265e2e42049bb67ff8e7c4096a', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:14.682447 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b355e4-e5fa-4113-8e0f-d60dd317cc29 tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:14.683751 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b355e4-e5fa-4113-8e0f-d60dd317cc29 tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:14.683751 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b355e4-e5fa-4113-8e0f-d60dd317cc29 tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] PATH_INFO: `/v3/projects/ff17367f68fe4b2eae56f38dde37575b` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:14.685109 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b355e4-e5fa-4113-8e0f-d60dd317cc29 tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] RBAC: Authorizing `identity:delete_project(project_id=ff17367f68fe4b2eae56f38dde37575b)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:14.687189 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b355e4-e5fa-4113-8e0f-d60dd317cc29 tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:14.844294 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-59b355e4-e5fa-4113-8e0f-d60dd317cc29 tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff17367f68fe4b2eae56f38dde37575b', 'request_id': 'req-59b355e4-e5fa-4113-8e0f-d60dd317cc29'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:14.917140 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-59b355e4-e5fa-4113-8e0f-d60dd317cc29 tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff17367f68fe4b2eae56f38dde37575b', 'request_id': 'req-59b355e4-e5fa-4113-8e0f-d60dd317cc29'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:14.924031 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-59b355e4-e5fa-4113-8e0f-d60dd317cc29 tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff17367f68fe4b2eae56f38dde37575b', 'request_id': 'req-59b355e4-e5fa-4113-8e0f-d60dd317cc29'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:14.935308 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-59b355e4-e5fa-4113-8e0f-d60dd317cc29 tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] The token cache is being invalidate because project ff17367f68fe4b2eae56f38dde37575b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:02:14.935596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-59b355e4-e5fa-4113-8e0f-d60dd317cc29 tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:14.936386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-59b355e4-e5fa-4113-8e0f-d60dd317cc29 tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:14.938211 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1251/2494] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:02:14 2022] DELETE /identity/v3/projects/ff17367f68fe4b2eae56f38dde37575b => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:15.100369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:15.814258 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097dda45-626b-42a8-9dbe-e509a07227ce tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:15.816164 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-097dda45-626b-42a8-9dbe-e509a07227ce tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:16.019763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097dda45-626b-42a8-9dbe-e509a07227ce tempest-VolumeQuotaClassesTest-510990927 tempest-VolumeQuotaClassesTest-510990927-project-admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:16.039567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:16.076070 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755492b7-df0e-4eff-9c6b-2ae117325feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:16.076711 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755492b7-df0e-4eff-9c6b-2ae117325feb admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:16.077085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755492b7-df0e-4eff-9c6b-2ae117325feb admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:16.077609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755492b7-df0e-4eff-9c6b-2ae117325feb admin admin] PATH_INFO: `/v3/users/474f367bbcf64cc48d0399ccd67df833` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:16.078878 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755492b7-df0e-4eff-9c6b-2ae117325feb admin admin] RBAC: Authorizing `identity:delete_user(user_id=474f367bbcf64cc48d0399ccd67df833)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:16.080832 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755492b7-df0e-4eff-9c6b-2ae117325feb admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:16.317907 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-755492b7-df0e-4eff-9c6b-2ae117325feb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '474f367bbcf64cc48d0399ccd67df833'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:16.318628 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-755492b7-df0e-4eff-9c6b-2ae117325feb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '474f367bbcf64cc48d0399ccd67df833'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:16.402738 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-755492b7-df0e-4eff-9c6b-2ae117325feb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '474f367bbcf64cc48d0399ccd67df833'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:16.411082 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-755492b7-df0e-4eff-9c6b-2ae117325feb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '474f367bbcf64cc48d0399ccd67df833'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:16.424363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-755492b7-df0e-4eff-9c6b-2ae117325feb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '474f367bbcf64cc48d0399ccd67df833'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:16.430516 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1244/2495] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:02:15 2022] DELETE /identity/v3/users/474f367bbcf64cc48d0399ccd67df833 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:16.435328 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d064f84-7aa9-4406-9d46-0c4ed8d3efa0 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:16.435896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d064f84-7aa9-4406-9d46-0c4ed8d3efa0 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:16.435997 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d064f84-7aa9-4406-9d46-0c4ed8d3efa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:16.457277 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-5d064f84-7aa9-4406-9d46-0c4ed8d3efa0 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:16.458535 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-5d064f84-7aa9-4406-9d46-0c4ed8d3efa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:16.518916 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1252/2496] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:16 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:16.529047 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755492b7-df0e-4eff-9c6b-2ae117325feb admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:16.530490 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-755492b7-df0e-4eff-9c6b-2ae117325feb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:16.587745 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755492b7-df0e-4eff-9c6b-2ae117325feb admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:16.591274 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02ef469-e6f5-4e34-8c68-96fd5134fd25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:16.591878 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ef469-e6f5-4e34-8c68-96fd5134fd25 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:16.592155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ef469-e6f5-4e34-8c68-96fd5134fd25 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:16.592457 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ef469-e6f5-4e34-8c68-96fd5134fd25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:16.601672 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02ef469-e6f5-4e34-8c68-96fd5134fd25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:16.603101 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02ef469-e6f5-4e34-8c68-96fd5134fd25 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:16.612561 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1245/2497] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:16 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:02:17.072903 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d064f84-7aa9-4406-9d46-0c4ed8d3efa0 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:17.074479 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-5d064f84-7aa9-4406-9d46-0c4ed8d3efa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:17.089737 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d064f84-7aa9-4406-9d46-0c4ed8d3efa0 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:17.095569 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0608267-1681-4a96-9488-efae0b691090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:17.096226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0608267-1681-4a96-9488-efae0b691090 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:17.096510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0608267-1681-4a96-9488-efae0b691090 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:17.096782 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0608267-1681-4a96-9488-efae0b691090 admin admin] PATH_INFO: `/v3/projects/b734c17e6b404d2dbce69eb4b4faf3da` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:17.097958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0608267-1681-4a96-9488-efae0b691090 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b734c17e6b404d2dbce69eb4b4faf3da)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:17.100035 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0608267-1681-4a96-9488-efae0b691090 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:17.296483 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-d0608267-1681-4a96-9488-efae0b691090 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b734c17e6b404d2dbce69eb4b4faf3da', 'request_id': 'req-d0608267-1681-4a96-9488-efae0b691090'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:17.415131 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-d0608267-1681-4a96-9488-efae0b691090 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b734c17e6b404d2dbce69eb4b4faf3da', 'request_id': 'req-d0608267-1681-4a96-9488-efae0b691090'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:17.422635 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-d0608267-1681-4a96-9488-efae0b691090 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b734c17e6b404d2dbce69eb4b4faf3da', 'request_id': 'req-d0608267-1681-4a96-9488-efae0b691090'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:17.434616 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-d0608267-1681-4a96-9488-efae0b691090 admin admin] The token cache is being invalidate because project b734c17e6b404d2dbce69eb4b4faf3da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:02:17.434906 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-d0608267-1681-4a96-9488-efae0b691090 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:17.435663 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-d0608267-1681-4a96-9488-efae0b691090 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:17.437367 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1253/2498] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:02:17 2022] DELETE /identity/v3/projects/b734c17e6b404d2dbce69eb4b4faf3da => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:17.441596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02ef469-e6f5-4e34-8c68-96fd5134fd25 service neutron] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:17.451575 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02ef469-e6f5-4e34-8c68-96fd5134fd25 service neutron] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:17.497099 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:17.511572 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890a213-f980-44dd-890d-76ad085aa969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:17.512216 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890a213-f980-44dd-890d-76ad085aa969 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:17.512489 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890a213-f980-44dd-890d-76ad085aa969 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:17.512760 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890a213-f980-44dd-890d-76ad085aa969 admin admin] PATH_INFO: `/v3/users/d71a54f8e0bc409a8a07d3da11cdbc70` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:17.513909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890a213-f980-44dd-890d-76ad085aa969 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d71a54f8e0bc409a8a07d3da11cdbc70)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:17.516781 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890a213-f980-44dd-890d-76ad085aa969 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:17.739846 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-2890a213-f980-44dd-890d-76ad085aa969 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd71a54f8e0bc409a8a07d3da11cdbc70'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:17.740397 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-2890a213-f980-44dd-890d-76ad085aa969 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd71a54f8e0bc409a8a07d3da11cdbc70'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:17.845273 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-2890a213-f980-44dd-890d-76ad085aa969 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd71a54f8e0bc409a8a07d3da11cdbc70'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:17.852859 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-2890a213-f980-44dd-890d-76ad085aa969 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd71a54f8e0bc409a8a07d3da11cdbc70'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:17.865867 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-2890a213-f980-44dd-890d-76ad085aa969 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd71a54f8e0bc409a8a07d3da11cdbc70'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:17.871846 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1246/2499] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:02:17 2022] DELETE /identity/v3/users/d71a54f8e0bc409a8a07d3da11cdbc70 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:18.418608 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0608267-1681-4a96-9488-efae0b691090 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:18.420297 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-d0608267-1681-4a96-9488-efae0b691090 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:18.435809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0608267-1681-4a96-9488-efae0b691090 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:18.501337 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4e757a-d28b-4942-8d2f-aba49f69b238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:18.501976 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e757a-d28b-4942-8d2f-aba49f69b238 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:18.502254 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e757a-d28b-4942-8d2f-aba49f69b238 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:18.502585 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4e757a-d28b-4942-8d2f-aba49f69b238 admin admin] PATH_INFO: `/v3/projects/5a320e265e2e42049bb67ff8e7c4096a` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:18.503821 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e757a-d28b-4942-8d2f-aba49f69b238 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a320e265e2e42049bb67ff8e7c4096a)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:18.505896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4e757a-d28b-4942-8d2f-aba49f69b238 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:18.612574 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:18.629415 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-da4e757a-d28b-4942-8d2f-aba49f69b238 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a320e265e2e42049bb67ff8e7c4096a', 'request_id': 'req-da4e757a-d28b-4942-8d2f-aba49f69b238'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:18.714049 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-da4e757a-d28b-4942-8d2f-aba49f69b238 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a320e265e2e42049bb67ff8e7c4096a', 'request_id': 'req-da4e757a-d28b-4942-8d2f-aba49f69b238'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:18.721274 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-da4e757a-d28b-4942-8d2f-aba49f69b238 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a320e265e2e42049bb67ff8e7c4096a', 'request_id': 'req-da4e757a-d28b-4942-8d2f-aba49f69b238'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:18.732660 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-da4e757a-d28b-4942-8d2f-aba49f69b238 admin admin] The token cache is being invalidate because project 5a320e265e2e42049bb67ff8e7c4096a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:02:18.733618 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-da4e757a-d28b-4942-8d2f-aba49f69b238 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:18.735592 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-da4e757a-d28b-4942-8d2f-aba49f69b238 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:18.735870 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1254/2500] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:02:18 2022] DELETE /identity/v3/projects/5a320e265e2e42049bb67ff8e7c4096a => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:18.741954 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f34792-c18c-4650-a544-e61674d58b68 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:18.742489 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f34792-c18c-4650-a544-e61674d58b68 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:18.742590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f34792-c18c-4650-a544-e61674d58b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:18.761940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-33f34792-c18c-4650-a544-e61674d58b68 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:18.763178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-33f34792-c18c-4650-a544-e61674d58b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:18.824776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1247/2501] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:18 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:18.832811 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301feb47-3b0d-48f9-aca6-7ec1d279215d None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:18.833091 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301feb47-3b0d-48f9-aca6-7ec1d279215d None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:18.833365 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301feb47-3b0d-48f9-aca6-7ec1d279215d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:18.854416 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-301feb47-3b0d-48f9-aca6-7ec1d279215d None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:18.855677 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-301feb47-3b0d-48f9-aca6-7ec1d279215d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:18.865064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1255/2502] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:18 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:18.869962 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f34792-c18c-4650-a544-e61674d58b68 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:18.880222 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f34792-c18c-4650-a544-e61674d58b68 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:18.885088 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47a35d-8964-41bd-8344-68fd46396f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:18.885697 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47a35d-8964-41bd-8344-68fd46396f1d admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:18.885973 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47a35d-8964-41bd-8344-68fd46396f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:18.886246 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a47a35d-8964-41bd-8344-68fd46396f1d admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:18.886938 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47a35d-8964-41bd-8344-68fd46396f1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:02:18.887282 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47a35d-8964-41bd-8344-68fd46396f1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:18.888848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a47a35d-8964-41bd-8344-68fd46396f1d admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:18.901267 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1248/2503] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 11:02:18 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:02:18.908585 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301feb47-3b0d-48f9-aca6-7ec1d279215d None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:18.917827 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301feb47-3b0d-48f9-aca6-7ec1d279215d None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:18.923388 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28044046-19b2-44ad-b77e-05597a67c47b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:18.923984 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28044046-19b2-44ad-b77e-05597a67c47b admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:18.924258 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28044046-19b2-44ad-b77e-05597a67c47b admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:18.924523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28044046-19b2-44ad-b77e-05597a67c47b admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:18.925181 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28044046-19b2-44ad-b77e-05597a67c47b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:18.927070 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28044046-19b2-44ad-b77e-05597a67c47b admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:19.028769 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1256/2504] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:02:18 2022] POST /identity/v3/projects => generated 378 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:19.033287 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47a35d-8964-41bd-8344-68fd46396f1d admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:19.041546 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a47a35d-8964-41bd-8344-68fd46396f1d admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:19.102848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:19.112579 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b79151b-0406-424e-a433-83e85753e8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:19.116747 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79151b-0406-424e-a433-83e85753e8f1 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:19.116747 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79151b-0406-424e-a433-83e85753e8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:19.116747 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b79151b-0406-424e-a433-83e85753e8f1 admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:19.116747 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b79151b-0406-424e-a433-83e85753e8f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:19.116747 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b79151b-0406-424e-a433-83e85753e8f1 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:19.188899 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1249/2505] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:02:19 2022] POST /identity/v3/users => generated 290 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:19.193022 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28044046-19b2-44ad-b77e-05597a67c47b admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:19.203946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28044046-19b2-44ad-b77e-05597a67c47b admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:19.208691 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25eebd5-e343-4a40-a809-998b233b1343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:19.209261 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25eebd5-e343-4a40-a809-998b233b1343 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:19.209538 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25eebd5-e343-4a40-a809-998b233b1343 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:19.209807 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25eebd5-e343-4a40-a809-998b233b1343 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:19.210487 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25eebd5-e343-4a40-a809-998b233b1343 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:19.212254 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25eebd5-e343-4a40-a809-998b233b1343 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:19.223462 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1257/2506] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:19 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:19.227543 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b79151b-0406-424e-a433-83e85753e8f1 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:19.238064 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b79151b-0406-424e-a433-83e85753e8f1 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:19.242665 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867677c0-36de-4abd-b465-f595b9879ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:19.243452 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867677c0-36de-4abd-b465-f595b9879ed8 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:19.243786 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867677c0-36de-4abd-b465-f595b9879ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:19.244072 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867677c0-36de-4abd-b465-f595b9879ed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:19.244801 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867677c0-36de-4abd-b465-f595b9879ed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:19.246397 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867677c0-36de-4abd-b465-f595b9879ed8 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:19.257726 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1250/2507] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:19 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:19.261681 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25eebd5-e343-4a40-a809-998b233b1343 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:19.272685 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25eebd5-e343-4a40-a809-998b233b1343 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:19.277520 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89021dc-bb25-49fa-a8d3-b026bea69349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:19.278502 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89021dc-bb25-49fa-a8d3-b026bea69349 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:19.278789 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89021dc-bb25-49fa-a8d3-b026bea69349 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:19.279146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89021dc-bb25-49fa-a8d3-b026bea69349 admin admin] PATH_INFO: `/v3/projects/38eae2393aa740d3af918c12d2ff1ec5/users/7d100f35ab534f9caedf4dda6ca259e9/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:19.298459 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89021dc-bb25-49fa-a8d3-b026bea69349 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38eae2393aa740d3af918c12d2ff1ec5, user_id=7d100f35ab534f9caedf4dda6ca259e9, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:19.301408 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89021dc-bb25-49fa-a8d3-b026bea69349 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:19.422635 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1258/2508] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:19 2022] PUT /identity/v3/projects/38eae2393aa740d3af918c12d2ff1ec5/users/7d100f35ab534f9caedf4dda6ca259e9/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:19.437478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad834c6-b9c2-4818-9c15-0338b4a30acd None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:19.437478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad834c6-b9c2-4818-9c15-0338b4a30acd None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:19.437478 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad834c6-b9c2-4818-9c15-0338b4a30acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:19.463676 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-1ad834c6-b9c2-4818-9c15-0338b4a30acd None None] MFA Rules not processed for user `7d100f35ab534f9caedf4dda6ca259e9`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:19.464870 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-1ad834c6-b9c2-4818-9c15-0338b4a30acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:19.535132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1251/2509] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:19 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:19.554667 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89021dc-bb25-49fa-a8d3-b026bea69349 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:19.556731 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-d89021dc-bb25-49fa-a8d3-b026bea69349 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:19.616957 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89021dc-bb25-49fa-a8d3-b026bea69349 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:19.620613 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab6d9ae-b963-465e-982c-f0a50e7bb9fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:19.621231 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6d9ae-b963-465e-982c-f0a50e7bb9fb service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:19.621510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6d9ae-b963-465e-982c-f0a50e7bb9fb service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:19.621784 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6d9ae-b963-465e-982c-f0a50e7bb9fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:19.630889 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab6d9ae-b963-465e-982c-f0a50e7bb9fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:19.632297 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab6d9ae-b963-465e-982c-f0a50e7bb9fb service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:19.642077 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1259/2510] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:19 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:02:20.421229 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad834c6-b9c2-4818-9c15-0338b4a30acd None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:20.424295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-1ad834c6-b9c2-4818-9c15-0338b4a30acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:20.481604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab6d9ae-b963-465e-982c-f0a50e7bb9fb service cinder] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:20.483642 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-bab6d9ae-b963-465e-982c-f0a50e7bb9fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:20.516720 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad834c6-b9c2-4818-9c15-0338b4a30acd None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:20.523083 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182a09f3-514b-46ad-9b62-471ec50a07a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:20.524621 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a09f3-514b-46ad-9b62-471ec50a07a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:20.524919 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a09f3-514b-46ad-9b62-471ec50a07a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:20.526819 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182a09f3-514b-46ad-9b62-471ec50a07a7 admin admin] PATH_INFO: `/v3/users/dd2ddb6627824799a2c1831fce7ac1ed` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:20.526819 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a09f3-514b-46ad-9b62-471ec50a07a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd2ddb6627824799a2c1831fce7ac1ed)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:20.528508 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182a09f3-514b-46ad-9b62-471ec50a07a7 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:20.544809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:20.559812 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab6d9ae-b963-465e-982c-f0a50e7bb9fb service cinder] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:20.565420 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cce356-cd8e-49a3-9eb8-f0f24078db37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:20.566064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cce356-cd8e-49a3-9eb8-f0f24078db37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:20.566340 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cce356-cd8e-49a3-9eb8-f0f24078db37 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:20.566610 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cce356-cd8e-49a3-9eb8-f0f24078db37 admin admin] PATH_INFO: `/v3/users/a039ffa0adf04575b5493c622735e5b7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:20.567859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cce356-cd8e-49a3-9eb8-f0f24078db37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a039ffa0adf04575b5493c622735e5b7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:20.569789 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cce356-cd8e-49a3-9eb8-f0f24078db37 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:20.766873 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-43cce356-cd8e-49a3-9eb8-f0f24078db37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a039ffa0adf04575b5493c622735e5b7'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:20.768185 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-182a09f3-514b-46ad-9b62-471ec50a07a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd2ddb6627824799a2c1831fce7ac1ed'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:20.769080 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-182a09f3-514b-46ad-9b62-471ec50a07a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd2ddb6627824799a2c1831fce7ac1ed'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:20.774499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-43cce356-cd8e-49a3-9eb8-f0f24078db37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a039ffa0adf04575b5493c622735e5b7'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:20.775457 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-43cce356-cd8e-49a3-9eb8-f0f24078db37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a039ffa0adf04575b5493c622735e5b7'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:20.842277 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-182a09f3-514b-46ad-9b62-471ec50a07a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd2ddb6627824799a2c1831fce7ac1ed'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:20.850848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-182a09f3-514b-46ad-9b62-471ec50a07a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd2ddb6627824799a2c1831fce7ac1ed'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:20.851967 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:20.863804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-182a09f3-514b-46ad-9b62-471ec50a07a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd2ddb6627824799a2c1831fce7ac1ed'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:20.869713 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1252/2511] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:02:20 2022] DELETE /identity/v3/users/dd2ddb6627824799a2c1831fce7ac1ed => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:20.873992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d3e04-abe1-4bed-9467-98d707e0ffc3 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:20.874278 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d3e04-abe1-4bed-9467-98d707e0ffc3 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:20.874553 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544d3e04-abe1-4bed-9467-98d707e0ffc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:20.892949 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-544d3e04-abe1-4bed-9467-98d707e0ffc3 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:20.894145 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-544d3e04-abe1-4bed-9467-98d707e0ffc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:20.906046 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-43cce356-cd8e-49a3-9eb8-f0f24078db37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a039ffa0adf04575b5493c622735e5b7'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:20.920467 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-43cce356-cd8e-49a3-9eb8-f0f24078db37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a039ffa0adf04575b5493c622735e5b7'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:20.927498 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1260/2512] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:02:20 2022] DELETE /identity/v3/users/a039ffa0adf04575b5493c622735e5b7 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:02:20.932110 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a25e339-e11f-498d-b005-d88680513838 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:20.932394 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a25e339-e11f-498d-b005-d88680513838 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:20.932668 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a25e339-e11f-498d-b005-d88680513838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:20.951544 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-9a25e339-e11f-498d-b005-d88680513838 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:20.952745 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-9a25e339-e11f-498d-b005-d88680513838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:20.954815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1253/2513] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:20 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:02:20.965369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d3e04-abe1-4bed-9467-98d707e0ffc3 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:20.970513 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-544d3e04-abe1-4bed-9467-98d707e0ffc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:21.012573 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1261/2514] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:20 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:02:21.022607 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a25e339-e11f-498d-b005-d88680513838 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:21.025465 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544d3e04-abe1-4bed-9467-98d707e0ffc3 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:21.077529 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dd9a5-6fee-4f0f-a569-1caf8e1be691 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:21.078156 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dd9a5-6fee-4f0f-a569-1caf8e1be691 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:21.078484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dd9a5-6fee-4f0f-a569-1caf8e1be691 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:21.078763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14dd9a5-6fee-4f0f-a569-1caf8e1be691 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:21.082250 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a25e339-e11f-498d-b005-d88680513838 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:21.085987 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b28b83-4960-4928-b11e-9ffe9d03ac55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:21.086600 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b28b83-4960-4928-b11e-9ffe9d03ac55 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:21.086878 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b28b83-4960-4928-b11e-9ffe9d03ac55 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:21.087178 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b28b83-4960-4928-b11e-9ffe9d03ac55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:21.088374 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14dd9a5-6fee-4f0f-a569-1caf8e1be691 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:21.089865 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14dd9a5-6fee-4f0f-a569-1caf8e1be691 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:21.097292 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b28b83-4960-4928-b11e-9ffe9d03ac55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:21.098725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b28b83-4960-4928-b11e-9ffe9d03ac55 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:21.099962 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1254/2515] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:20 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:02:21.110854 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1262/2516] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:21 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:02:21.712934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dd9a5-6fee-4f0f-a569-1caf8e1be691 service neutron] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:21.713581 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b28b83-4960-4928-b11e-9ffe9d03ac55 service neutron] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:21.714067 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-e14dd9a5-6fee-4f0f-a569-1caf8e1be691 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:21.714263 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f2b28b83-4960-4928-b11e-9ffe9d03ac55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:21.729522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b28b83-4960-4928-b11e-9ffe9d03ac55 service neutron] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:21.736417 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e37d8-1fbb-4a77-815e-8de7b4853a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:21.738483 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e37d8-1fbb-4a77-815e-8de7b4853a73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:21.739177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e37d8-1fbb-4a77-815e-8de7b4853a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:21.739622 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4e37d8-1fbb-4a77-815e-8de7b4853a73 admin admin] PATH_INFO: `/v3/projects/f022e65418144d8b9e205985cfba0df7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:21.742426 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e37d8-1fbb-4a77-815e-8de7b4853a73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f022e65418144d8b9e205985cfba0df7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:21.744580 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14dd9a5-6fee-4f0f-a569-1caf8e1be691 service neutron] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:21.747387 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4e37d8-1fbb-4a77-815e-8de7b4853a73 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:21.755190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f9d22-0dd0-4a14-b6be-59852eb8c81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:21.756598 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f9d22-0dd0-4a14-b6be-59852eb8c81a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:21.757730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f9d22-0dd0-4a14-b6be-59852eb8c81a admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:21.758247 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f9d22-0dd0-4a14-b6be-59852eb8c81a admin admin] PATH_INFO: `/v3/projects/a7b6e1a86a7e4876a222b007067c94ff` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:21.759918 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f9d22-0dd0-4a14-b6be-59852eb8c81a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7b6e1a86a7e4876a222b007067c94ff)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:21.762966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f9d22-0dd0-4a14-b6be-59852eb8c81a admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:21.765631 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:21.877765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:21.886058 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-9b4e37d8-1fbb-4a77-815e-8de7b4853a73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f022e65418144d8b9e205985cfba0df7', 'request_id': 'req-9b4e37d8-1fbb-4a77-815e-8de7b4853a73'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:21.887469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-938f9d22-0dd0-4a14-b6be-59852eb8c81a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7b6e1a86a7e4876a222b007067c94ff', 'request_id': 'req-938f9d22-0dd0-4a14-b6be-59852eb8c81a'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:21.894361 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-938f9d22-0dd0-4a14-b6be-59852eb8c81a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7b6e1a86a7e4876a222b007067c94ff', 'request_id': 'req-938f9d22-0dd0-4a14-b6be-59852eb8c81a'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:21.995024 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-9b4e37d8-1fbb-4a77-815e-8de7b4853a73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f022e65418144d8b9e205985cfba0df7', 'request_id': 'req-9b4e37d8-1fbb-4a77-815e-8de7b4853a73'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:22.001976 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-9b4e37d8-1fbb-4a77-815e-8de7b4853a73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f022e65418144d8b9e205985cfba0df7', 'request_id': 'req-9b4e37d8-1fbb-4a77-815e-8de7b4853a73'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:22.014977 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-9b4e37d8-1fbb-4a77-815e-8de7b4853a73 admin admin] The token cache is being invalidate because project f022e65418144d8b9e205985cfba0df7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:02:22.015276 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-9b4e37d8-1fbb-4a77-815e-8de7b4853a73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:22.016533 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-9b4e37d8-1fbb-4a77-815e-8de7b4853a73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:22.018087 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1263/2517] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:02:21 2022] DELETE /identity/v3/projects/f022e65418144d8b9e205985cfba0df7 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:22.022610 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e37d8-1fbb-4a77-815e-8de7b4853a73 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:22.024345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-9b4e37d8-1fbb-4a77-815e-8de7b4853a73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:22.065190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-938f9d22-0dd0-4a14-b6be-59852eb8c81a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7b6e1a86a7e4876a222b007067c94ff', 'request_id': 'req-938f9d22-0dd0-4a14-b6be-59852eb8c81a'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:22.077583 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-938f9d22-0dd0-4a14-b6be-59852eb8c81a admin admin] The token cache is being invalidate because project a7b6e1a86a7e4876a222b007067c94ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:02:22.077875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-938f9d22-0dd0-4a14-b6be-59852eb8c81a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:22.079179 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-938f9d22-0dd0-4a14-b6be-59852eb8c81a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:22.080754 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1255/2518] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:02:21 2022] DELETE /identity/v3/projects/a7b6e1a86a7e4876a222b007067c94ff => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:22.084692 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f9d22-0dd0-4a14-b6be-59852eb8c81a admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:22.085855 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-938f9d22-0dd0-4a14-b6be-59852eb8c81a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:22.091314 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4e37d8-1fbb-4a77-815e-8de7b4853a73 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:22.151157 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82c656-e8a8-46af-bc00-cc605765b327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:22.151157 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82c656-e8a8-46af-bc00-cc605765b327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:22.151157 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82c656-e8a8-46af-bc00-cc605765b327 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:22.151157 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82c656-e8a8-46af-bc00-cc605765b327 admin admin] PATH_INFO: `/v3/users/c04f8f995bde4f3196c7959c199ee885` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:22.154085 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b82c656-e8a8-46af-bc00-cc605765b327 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c04f8f995bde4f3196c7959c199ee885)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:22.154085 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b82c656-e8a8-46af-bc00-cc605765b327 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:22.157300 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f9d22-0dd0-4a14-b6be-59852eb8c81a admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:22.163264 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5b0e0-031c-40ee-88c4-c6706e6e6fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:22.163908 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5b0e0-031c-40ee-88c4-c6706e6e6fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:22.164251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5b0e0-031c-40ee-88c4-c6706e6e6fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:22.164533 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd5b0e0-031c-40ee-88c4-c6706e6e6fb6 admin admin] PATH_INFO: `/v3/users/31db839d54544ddd990cfc74f53b436a` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:22.165741 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd5b0e0-031c-40ee-88c4-c6706e6e6fb6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31db839d54544ddd990cfc74f53b436a)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:22.167741 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd5b0e0-031c-40ee-88c4-c6706e6e6fb6 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:22.434657 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-dbd5b0e0-031c-40ee-88c4-c6706e6e6fb6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31db839d54544ddd990cfc74f53b436a'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:22.435398 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-dbd5b0e0-031c-40ee-88c4-c6706e6e6fb6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31db839d54544ddd990cfc74f53b436a'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:22.442362 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2b82c656-e8a8-46af-bc00-cc605765b327 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c04f8f995bde4f3196c7959c199ee885'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:22.443610 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:22.450145 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2b82c656-e8a8-46af-bc00-cc605765b327 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c04f8f995bde4f3196c7959c199ee885'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:22.450859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2b82c656-e8a8-46af-bc00-cc605765b327 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c04f8f995bde4f3196c7959c199ee885'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:22.520000 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-dbd5b0e0-031c-40ee-88c4-c6706e6e6fb6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31db839d54544ddd990cfc74f53b436a'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:22.528275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-dbd5b0e0-031c-40ee-88c4-c6706e6e6fb6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31db839d54544ddd990cfc74f53b436a'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:22.540944 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-dbd5b0e0-031c-40ee-88c4-c6706e6e6fb6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31db839d54544ddd990cfc74f53b436a'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:22.546856 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1256/2519] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:02:22 2022] DELETE /identity/v3/users/31db839d54544ddd990cfc74f53b436a => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:02:22.645831 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2b82c656-e8a8-46af-bc00-cc605765b327 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c04f8f995bde4f3196c7959c199ee885'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:22.659693 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2b82c656-e8a8-46af-bc00-cc605765b327 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c04f8f995bde4f3196c7959c199ee885'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:22.665552 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1264/2520] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:02:22 2022] DELETE /identity/v3/users/c04f8f995bde4f3196c7959c199ee885 => generated 0 bytes in 644 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:02:23.038314 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5b0e0-031c-40ee-88c4-c6706e6e6fb6 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:23.039897 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-dbd5b0e0-031c-40ee-88c4-c6706e6e6fb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:23.089037 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:23.108509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd5b0e0-031c-40ee-88c4-c6706e6e6fb6 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:23.135558 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82c656-e8a8-46af-bc00-cc605765b327 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:23.137826 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-2b82c656-e8a8-46af-bc00-cc605765b327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:23.175577 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b916c998-80a4-4b46-8375-694c42daaefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:23.176322 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916c998-80a4-4b46-8375-694c42daaefd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:23.176604 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916c998-80a4-4b46-8375-694c42daaefd admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:23.176884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b916c998-80a4-4b46-8375-694c42daaefd admin admin] PATH_INFO: `/v3/projects/f2c1c2f4f3184068928d3155115b37ae` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:23.178077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b916c998-80a4-4b46-8375-694c42daaefd admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2c1c2f4f3184068928d3155115b37ae)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:23.180162 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b916c998-80a4-4b46-8375-694c42daaefd admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:23.211484 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82c656-e8a8-46af-bc00-cc605765b327 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:23.216242 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b425ada-d9dc-44f0-9529-90aa6a2defae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:23.216892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b425ada-d9dc-44f0-9529-90aa6a2defae admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:23.217172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b425ada-d9dc-44f0-9529-90aa6a2defae admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:23.217450 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b425ada-d9dc-44f0-9529-90aa6a2defae admin admin] PATH_INFO: `/v3/projects/647135436c91467db9336ff84482b712` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:23.218651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b425ada-d9dc-44f0-9529-90aa6a2defae admin admin] RBAC: Authorizing `identity:delete_project(project_id=647135436c91467db9336ff84482b712)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:23.221391 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b425ada-d9dc-44f0-9529-90aa6a2defae admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:23.330779 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-b916c998-80a4-4b46-8375-694c42daaefd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2c1c2f4f3184068928d3155115b37ae', 'request_id': 'req-b916c998-80a4-4b46-8375-694c42daaefd'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:23.336806 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-b916c998-80a4-4b46-8375-694c42daaefd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2c1c2f4f3184068928d3155115b37ae', 'request_id': 'req-b916c998-80a4-4b46-8375-694c42daaefd'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:23.358249 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-7b425ada-d9dc-44f0-9529-90aa6a2defae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '647135436c91467db9336ff84482b712', 'request_id': 'req-7b425ada-d9dc-44f0-9529-90aa6a2defae'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:23.428287 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-b916c998-80a4-4b46-8375-694c42daaefd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2c1c2f4f3184068928d3155115b37ae', 'request_id': 'req-b916c998-80a4-4b46-8375-694c42daaefd'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:23.442414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-b916c998-80a4-4b46-8375-694c42daaefd admin admin] The token cache is being invalidate because project f2c1c2f4f3184068928d3155115b37ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:02:23.442704 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-b916c998-80a4-4b46-8375-694c42daaefd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:23.443427 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-b916c998-80a4-4b46-8375-694c42daaefd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:23.445003 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1257/2521] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:02:23 2022] DELETE /identity/v3/projects/f2c1c2f4f3184068928d3155115b37ae => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:23.449576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b0a96-ea3a-4e4f-8144-b41bf034570a None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:23.449862 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b0a96-ea3a-4e4f-8144-b41bf034570a None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:23.450140 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16b0a96-ea3a-4e4f-8144-b41bf034570a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:23.470375 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-a16b0a96-ea3a-4e4f-8144-b41bf034570a None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:23.472571 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-a16b0a96-ea3a-4e4f-8144-b41bf034570a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:23.494002 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-7b425ada-d9dc-44f0-9529-90aa6a2defae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '647135436c91467db9336ff84482b712', 'request_id': 'req-7b425ada-d9dc-44f0-9529-90aa6a2defae'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:23.500455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-7b425ada-d9dc-44f0-9529-90aa6a2defae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '647135436c91467db9336ff84482b712', 'request_id': 'req-7b425ada-d9dc-44f0-9529-90aa6a2defae'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:23.513814 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-7b425ada-d9dc-44f0-9529-90aa6a2defae admin admin] The token cache is being invalidate because project 647135436c91467db9336ff84482b712 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:02:23.514140 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-7b425ada-d9dc-44f0-9529-90aa6a2defae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:23.514791 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-7b425ada-d9dc-44f0-9529-90aa6a2defae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:23.516340 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1265/2522] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:02:23 2022] DELETE /identity/v3/projects/647135436c91467db9336ff84482b712 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:23.523064 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe6859-2a5c-4568-9e10-a1e20484fb51 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:23.523351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe6859-2a5c-4568-9e10-a1e20484fb51 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:23.523936 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe6859-2a5c-4568-9e10-a1e20484fb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:23.535034 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1258/2523] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:23 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:02:23.544627 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6240f4-f1d8-49c8-ac46-813452f25222 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:23.544802 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-2cfe6859-2a5c-4568-9e10-a1e20484fb51 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:23.544912 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6240f4-f1d8-49c8-ac46-813452f25222 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:23.545185 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6240f4-f1d8-49c8-ac46-813452f25222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:23.545870 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-2cfe6859-2a5c-4568-9e10-a1e20484fb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:23.554760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1266/2524] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:23 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:02:23.566013 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-df6240f4-f1d8-49c8-ac46-813452f25222 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:23.567377 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-df6240f4-f1d8-49c8-ac46-813452f25222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:23.575987 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1259/2525] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:23 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:02:23.581108 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe6859-2a5c-4568-9e10-a1e20484fb51 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:23.591852 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe6859-2a5c-4568-9e10-a1e20484fb51 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:23.597540 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4e9747-d13a-4197-bdec-0ded72791ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:23.598188 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e9747-d13a-4197-bdec-0ded72791ad0 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:23.598466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e9747-d13a-4197-bdec-0ded72791ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:23.598741 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4e9747-d13a-4197-bdec-0ded72791ad0 admin admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:23.599495 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e9747-d13a-4197-bdec-0ded72791ad0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:02:23.599801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e9747-d13a-4197-bdec-0ded72791ad0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:23.601515 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4e9747-d13a-4197-bdec-0ded72791ad0 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:23.618629 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1267/2526] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 11:02:23 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:02:23.619789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6240f4-f1d8-49c8-ac46-813452f25222 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:23.626444 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a65ba-ea8f-4112-92ed-44b87d50cab2 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:23.626735 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a65ba-ea8f-4112-92ed-44b87d50cab2 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:23.627229 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344a65ba-ea8f-4112-92ed-44b87d50cab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:23.629856 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6240f4-f1d8-49c8-ac46-813452f25222 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:23.635136 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b1ecf8-678c-440c-b554-683ee41fd6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:23.635754 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1ecf8-678c-440c-b554-683ee41fd6c7 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:23.636037 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1ecf8-678c-440c-b554-683ee41fd6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:23.636317 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1ecf8-678c-440c-b554-683ee41fd6c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:23.636993 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b1ecf8-678c-440c-b554-683ee41fd6c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:23.638951 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b1ecf8-678c-440c-b554-683ee41fd6c7 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:23.650594 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-344a65ba-ea8f-4112-92ed-44b87d50cab2 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:23.651911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-344a65ba-ea8f-4112-92ed-44b87d50cab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:23.660858 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1268/2527] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:23 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:02:23.665031 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a65ba-ea8f-4112-92ed-44b87d50cab2 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:23.678079 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344a65ba-ea8f-4112-92ed-44b87d50cab2 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:23.685541 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb7d4e-c95e-47fb-b69b-e790271345ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:23.686429 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb7d4e-c95e-47fb-b69b-e790271345ac admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:23.686665 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb7d4e-c95e-47fb-b69b-e790271345ac admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:23.686971 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceb7d4e-c95e-47fb-b69b-e790271345ac admin admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:23.687801 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb7d4e-c95e-47fb-b69b-e790271345ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:02:23.688146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb7d4e-c95e-47fb-b69b-e790271345ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:23.690934 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceb7d4e-c95e-47fb-b69b-e790271345ac admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:23.706634 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1269/2528] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 11:02:23 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 11:02:23.711567 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb7d4e-c95e-47fb-b69b-e790271345ac admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:23.724143 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceb7d4e-c95e-47fb-b69b-e790271345ac admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:23.728648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b33af44-79ca-46c2-82c4-1cc1cd8a8f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:23.729272 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33af44-79ca-46c2-82c4-1cc1cd8a8f24 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:23.729653 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33af44-79ca-46c2-82c4-1cc1cd8a8f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:23.729941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b33af44-79ca-46c2-82c4-1cc1cd8a8f24 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:23.730611 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b33af44-79ca-46c2-82c4-1cc1cd8a8f24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:23.732655 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b33af44-79ca-46c2-82c4-1cc1cd8a8f24 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:23.747606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1260/2529] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:02:23 2022] POST /identity/v3/projects => generated 382 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:02:23.749492 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:23.753043 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b1ecf8-678c-440c-b554-683ee41fd6c7 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:23.816303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b1ecf8-678c-440c-b554-683ee41fd6c7 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:23.821344 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e2137-5cb8-4c6d-bd72-89cab55032d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:23.821964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e2137-5cb8-4c6d-bd72-89cab55032d2 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:23.822238 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e2137-5cb8-4c6d-bd72-89cab55032d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:23.822515 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e2137-5cb8-4c6d-bd72-89cab55032d2 admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:23.823207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e2137-5cb8-4c6d-bd72-89cab55032d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:23.825315 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e2137-5cb8-4c6d-bd72-89cab55032d2 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:23.863905 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1270/2530] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:02:23 2022] POST /identity/v3/projects => generated 384 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:02:23.870027 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b33af44-79ca-46c2-82c4-1cc1cd8a8f24 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:23.942516 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b33af44-79ca-46c2-82c4-1cc1cd8a8f24 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:23.947544 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a08bd-16e4-4b60-a6bc-321ba9acc72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:23.948142 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a08bd-16e4-4b60-a6bc-321ba9acc72b admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:23.948428 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a08bd-16e4-4b60-a6bc-321ba9acc72b admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:23.948717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4a08bd-16e4-4b60-a6bc-321ba9acc72b admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:23.949384 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a08bd-16e4-4b60-a6bc-321ba9acc72b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:23.951619 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4a08bd-16e4-4b60-a6bc-321ba9acc72b admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:23.954817 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1261/2531] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:02:23 2022] POST /identity/v3/users => generated 292 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:02:23.959824 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e2137-5cb8-4c6d-bd72-89cab55032d2 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:23.970065 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e2137-5cb8-4c6d-bd72-89cab55032d2 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:23.975027 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc24422-1add-43db-a93d-3d55964c37c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:23.975768 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc24422-1add-43db-a93d-3d55964c37c7 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:23.976045 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc24422-1add-43db-a93d-3d55964c37c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:23.976342 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc24422-1add-43db-a93d-3d55964c37c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:23.977065 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc24422-1add-43db-a93d-3d55964c37c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:23.978701 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc24422-1add-43db-a93d-3d55964c37c7 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:23.989340 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1262/2532] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:23 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:02:23.993426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc24422-1add-43db-a93d-3d55964c37c7 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:24.004653 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc24422-1add-43db-a93d-3d55964c37c7 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:24.009458 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874e2624-9c6b-4be6-a34e-cf667d6d738f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:24.010119 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e2624-9c6b-4be6-a34e-cf667d6d738f admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:24.010400 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e2624-9c6b-4be6-a34e-cf667d6d738f admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:24.010674 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874e2624-9c6b-4be6-a34e-cf667d6d738f admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:24.011453 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874e2624-9c6b-4be6-a34e-cf667d6d738f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:24.013052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874e2624-9c6b-4be6-a34e-cf667d6d738f admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:24.025006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1263/2533] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:23 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:02:24.029683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874e2624-9c6b-4be6-a34e-cf667d6d738f admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:24.041221 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874e2624-9c6b-4be6-a34e-cf667d6d738f admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:24.045953 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c488b95f-67b8-4f0f-92ff-6da2b9904429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:24.046701 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488b95f-67b8-4f0f-92ff-6da2b9904429 admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:24.046981 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488b95f-67b8-4f0f-92ff-6da2b9904429 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:24.047413 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c488b95f-67b8-4f0f-92ff-6da2b9904429 admin admin] PATH_INFO: `/v3/projects/2548330e0145481ca7157bd2d138e902/users/09b552743cc2456791cd43550bcab466/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:24.060099 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1271/2534] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:02:23 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:02:24.064228 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a08bd-16e4-4b60-a6bc-321ba9acc72b admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:24.065287 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488b95f-67b8-4f0f-92ff-6da2b9904429 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2548330e0145481ca7157bd2d138e902, user_id=09b552743cc2456791cd43550bcab466, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:24.067729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c488b95f-67b8-4f0f-92ff-6da2b9904429 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:24.074869 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4a08bd-16e4-4b60-a6bc-321ba9acc72b admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:24.079687 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902596b-cbe7-4ff0-a5bc-b7a6c4880bce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:24.081120 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902596b-cbe7-4ff0-a5bc-b7a6c4880bce admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:24.081371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902596b-cbe7-4ff0-a5bc-b7a6c4880bce admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:24.081646 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b902596b-cbe7-4ff0-a5bc-b7a6c4880bce admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:24.082355 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b902596b-cbe7-4ff0-a5bc-b7a6c4880bce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:24.083968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b902596b-cbe7-4ff0-a5bc-b7a6c4880bce admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:24.094100 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1272/2535] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:24 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:02:24.099985 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902596b-cbe7-4ff0-a5bc-b7a6c4880bce admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:24.109626 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b902596b-cbe7-4ff0-a5bc-b7a6c4880bce admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:24.114959 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baca18f-68a8-4e53-bd54-549eb473c091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:24.115589 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baca18f-68a8-4e53-bd54-549eb473c091 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:24.115859 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baca18f-68a8-4e53-bd54-549eb473c091 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:24.116132 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baca18f-68a8-4e53-bd54-549eb473c091 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:24.116950 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baca18f-68a8-4e53-bd54-549eb473c091 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:24.118523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baca18f-68a8-4e53-bd54-549eb473c091 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:24.129824 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1273/2536] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:24 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:02:24.134192 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baca18f-68a8-4e53-bd54-549eb473c091 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:24.144019 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baca18f-68a8-4e53-bd54-549eb473c091 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:24.149323 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f2d4e-128f-4072-8902-6c8a3a5433ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:24.149982 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f2d4e-128f-4072-8902-6c8a3a5433ac admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:24.150263 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f2d4e-128f-4072-8902-6c8a3a5433ac admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:24.150539 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f2d4e-128f-4072-8902-6c8a3a5433ac admin admin] PATH_INFO: `/v3/projects/b397060521564f4cb278d81e397d1988/users/010b7165fd5b45e7877f5365db25abe8/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:24.151810 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1264/2537] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:24 2022] PUT /identity/v3/projects/2548330e0145481ca7157bd2d138e902/users/09b552743cc2456791cd43550bcab466/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:02:24.163400 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842fd13-8409-4356-aff7-d921f68af377 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:24.163688 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842fd13-8409-4356-aff7-d921f68af377 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:24.163976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6842fd13-8409-4356-aff7-d921f68af377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:24.166887 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f2d4e-128f-4072-8902-6c8a3a5433ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=b397060521564f4cb278d81e397d1988, user_id=010b7165fd5b45e7877f5365db25abe8, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:24.169561 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f2d4e-128f-4072-8902-6c8a3a5433ac admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:24.174311 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:24.194300 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-6842fd13-8409-4356-aff7-d921f68af377 None None] MFA Rules not processed for user `09b552743cc2456791cd43550bcab466`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:24.195647 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-6842fd13-8409-4356-aff7-d921f68af377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:24.248590 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1274/2538] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:24 2022] PUT /identity/v3/projects/b397060521564f4cb278d81e397d1988/users/010b7165fd5b45e7877f5365db25abe8/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:02:24.260704 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac114cc-0209-409d-ae8d-6ff60fa4ca7a None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:24.260847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1265/2539] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:24 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:24.261176 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac114cc-0209-409d-ae8d-6ff60fa4ca7a None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:24.261552 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac114cc-0209-409d-ae8d-6ff60fa4ca7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:24.265353 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842fd13-8409-4356-aff7-d921f68af377 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:24.293188 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-6ac114cc-0209-409d-ae8d-6ff60fa4ca7a None None] MFA Rules not processed for user `010b7165fd5b45e7877f5365db25abe8`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:24.294486 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-6ac114cc-0209-409d-ae8d-6ff60fa4ca7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:24.331253 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6842fd13-8409-4356-aff7-d921f68af377 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:24.367099 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1275/2540] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:24.380705 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac114cc-0209-409d-ae8d-6ff60fa4ca7a None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:24.382137 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-6ac114cc-0209-409d-ae8d-6ff60fa4ca7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:24.395446 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4c867-dfab-4816-b38a-52ce2ad5c7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:24.396071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4c867-dfab-4816-b38a-52ce2ad5c7d2 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:24.396350 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4c867-dfab-4816-b38a-52ce2ad5c7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:24.396874 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b4c867-dfab-4816-b38a-52ce2ad5c7d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:24.397287 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b4c867-dfab-4816-b38a-52ce2ad5c7d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:24.399510 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b4c867-dfab-4816-b38a-52ce2ad5c7d2 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:24.417401 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:24.434387 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:24.446570 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac114cc-0209-409d-ae8d-6ff60fa4ca7a None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:24.450215 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d26a2-8e81-4263-97ec-c3aa4581a0af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:24.450830 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d26a2-8e81-4263-97ec-c3aa4581a0af service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:24.451137 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d26a2-8e81-4263-97ec-c3aa4581a0af service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:24.451413 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0d26a2-8e81-4263-97ec-c3aa4581a0af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:24.460150 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d26a2-8e81-4263-97ec-c3aa4581a0af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:24.461736 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0d26a2-8e81-4263-97ec-c3aa4581a0af service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:24.470692 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1276/2541] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:24 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:02:24.557956 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1266/2542] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:02:24 2022] POST /identity/v3/projects => generated 384 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:02:24.561701 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d26a2-8e81-4263-97ec-c3aa4581a0af service cinder] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:24.572582 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0d26a2-8e81-4263-97ec-c3aa4581a0af service cinder] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:24.630664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26741b8-406d-4118-8bed-4740ff794523 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:24.631319 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26741b8-406d-4118-8bed-4740ff794523 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:24.631591 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26741b8-406d-4118-8bed-4740ff794523 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:24.631868 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26741b8-406d-4118-8bed-4740ff794523 admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:24.632542 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26741b8-406d-4118-8bed-4740ff794523 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:24.634478 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26741b8-406d-4118-8bed-4740ff794523 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:24.724725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1277/2543] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:02:24 2022] POST /identity/v3/users => generated 299 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:24.728120 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4c867-dfab-4816-b38a-52ce2ad5c7d2 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:24.740280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b4c867-dfab-4816-b38a-52ce2ad5c7d2 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:24.745565 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dfb534-f215-4fa4-b79b-0d26c4d01c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:24.746254 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dfb534-f215-4fa4-b79b-0d26c4d01c03 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:24.746611 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dfb534-f215-4fa4-b79b-0d26c4d01c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:24.746955 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dfb534-f215-4fa4-b79b-0d26c4d01c03 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:24.747740 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dfb534-f215-4fa4-b79b-0d26c4d01c03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:24.749392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69dfb534-f215-4fa4-b79b-0d26c4d01c03 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:24.761066 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1267/2544] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:24 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:24.765893 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26741b8-406d-4118-8bed-4740ff794523 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:24.776494 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26741b8-406d-4118-8bed-4740ff794523 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:24.781354 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1890b6d5-02b4-487c-8169-c112176debe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:24.782089 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890b6d5-02b4-487c-8169-c112176debe3 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:24.782509 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890b6d5-02b4-487c-8169-c112176debe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:24.782891 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1890b6d5-02b4-487c-8169-c112176debe3 admin admin] PATH_INFO: `/v3/projects/da7d85e2aad04693bf45a468c6cb113f/users/ca718594ffea4b51b143a91ddf7961cc/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:24.802731 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1890b6d5-02b4-487c-8169-c112176debe3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da7d85e2aad04693bf45a468c6cb113f, user_id=ca718594ffea4b51b143a91ddf7961cc, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:24.805589 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1890b6d5-02b4-487c-8169-c112176debe3 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:24.889046 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1278/2545] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:24 2022] PUT /identity/v3/projects/da7d85e2aad04693bf45a468c6cb113f/users/ca718594ffea4b51b143a91ddf7961cc/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:02:24.893779 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dfb534-f215-4fa4-b79b-0d26c4d01c03 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:24.903947 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dfb534-f215-4fa4-b79b-0d26c4d01c03 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:24.969288 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71980b18-d99f-4d69-976e-201f946d864f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:24.970256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71980b18-d99f-4d69-976e-201f946d864f admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:24.970757 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71980b18-d99f-4d69-976e-201f946d864f admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:24.971308 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71980b18-d99f-4d69-976e-201f946d864f admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:24.973545 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71980b18-d99f-4d69-976e-201f946d864f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:24.975033 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71980b18-d99f-4d69-976e-201f946d864f admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:24.992127 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1268/2546] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:24 2022] GET /identity/v3/roles => generated 1427 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:24.996446 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1890b6d5-02b4-487c-8169-c112176debe3 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:25.007692 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1890b6d5-02b4-487c-8169-c112176debe3 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:25.012322 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3046f0c9-1f31-4847-bb0d-9d87135acd84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:25.012916 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3046f0c9-1f31-4847-bb0d-9d87135acd84 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:25.013195 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3046f0c9-1f31-4847-bb0d-9d87135acd84 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:25.013468 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3046f0c9-1f31-4847-bb0d-9d87135acd84 admin admin] PATH_INFO: `/v3/projects/da7d85e2aad04693bf45a468c6cb113f/users/ca718594ffea4b51b143a91ddf7961cc/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:25.015429 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3046f0c9-1f31-4847-bb0d-9d87135acd84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da7d85e2aad04693bf45a468c6cb113f, user_id=ca718594ffea4b51b143a91ddf7961cc, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:25.018105 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3046f0c9-1f31-4847-bb0d-9d87135acd84 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:25.028846 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1279/2547] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:24 2022] PUT /identity/v3/projects/da7d85e2aad04693bf45a468c6cb113f/users/ca718594ffea4b51b143a91ddf7961cc/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:25.039449 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ce728-18c4-4b02-8613-8018349acd03 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:25.044056 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ce728-18c4-4b02-8613-8018349acd03 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:25.044056 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7ce728-18c4-4b02-8613-8018349acd03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:25.069052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-dd7ce728-18c4-4b02-8613-8018349acd03 None None] MFA Rules not processed for user `ca718594ffea4b51b143a91ddf7961cc`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:25.070381 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-dd7ce728-18c4-4b02-8613-8018349acd03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:25.100489 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:25.139309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1269/2548] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:25 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:25.150841 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3046f0c9-1f31-4847-bb0d-9d87135acd84 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:25.152596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-3046f0c9-1f31-4847-bb0d-9d87135acd84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:25.218131 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3046f0c9-1f31-4847-bb0d-9d87135acd84 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:25.222130 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629de21-2150-4dbe-bb46-855853b33354 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:25.222762 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629de21-2150-4dbe-bb46-855853b33354 service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:25.223174 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629de21-2150-4dbe-bb46-855853b33354 service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:25.223339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8629de21-2150-4dbe-bb46-855853b33354 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:25.233452 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629de21-2150-4dbe-bb46-855853b33354 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:25.235221 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8629de21-2150-4dbe-bb46-855853b33354 service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:25.245913 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1280/2549] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:25 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:02:25.353118 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ce728-18c4-4b02-8613-8018349acd03 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:25.363503 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7ce728-18c4-4b02-8613-8018349acd03 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:25.367024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92671bcb-ac8f-4523-8ab7-a98099118496 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:25.367666 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92671bcb-ac8f-4523-8ab7-a98099118496 service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:25.367951 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92671bcb-ac8f-4523-8ab7-a98099118496 service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:25.368227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92671bcb-ac8f-4523-8ab7-a98099118496 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:25.426457 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92671bcb-ac8f-4523-8ab7-a98099118496 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:25.427916 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92671bcb-ac8f-4523-8ab7-a98099118496 service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:25.437354 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1270/2550] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:25 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:02:32.121160 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629de21-2150-4dbe-bb46-855853b33354 service cinder] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:32.130001 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8629de21-2150-4dbe-bb46-855853b33354 service cinder] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:32.133614 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff17ac8d-67b7-43b1-9273-8205c7b3753f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:32.134251 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17ac8d-67b7-43b1-9273-8205c7b3753f service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:32.134518 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17ac8d-67b7-43b1-9273-8205c7b3753f service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:32.134785 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff17ac8d-67b7-43b1-9273-8205c7b3753f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:32.136626 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-ff17ac8d-67b7-43b1-9273-8205c7b3753f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:32.158672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:32.196891 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff17ac8d-67b7-43b1-9273-8205c7b3753f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:32.198336 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff17ac8d-67b7-43b1-9273-8205c7b3753f service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:32.208111 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1281/2551] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:32 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:02:33.885256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92671bcb-ac8f-4523-8ab7-a98099118496 service cinder] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:33.948946 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92671bcb-ac8f-4523-8ab7-a98099118496 service cinder] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:33.953499 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b59eb-b0bf-4df6-bbc0-c59624df31cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:33.954145 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b59eb-b0bf-4df6-bbc0-c59624df31cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:33.954431 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b59eb-b0bf-4df6-bbc0-c59624df31cb admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:33.954711 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9b59eb-b0bf-4df6-bbc0-c59624df31cb admin admin] PATH_INFO: `/v3/users/09b552743cc2456791cd43550bcab466` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:33.955992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b59eb-b0bf-4df6-bbc0-c59624df31cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=09b552743cc2456791cd43550bcab466)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:33.957920 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9b59eb-b0bf-4df6-bbc0-c59624df31cb admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:34.235632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-7a9b59eb-b0bf-4df6-bbc0-c59624df31cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09b552743cc2456791cd43550bcab466'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:34.237036 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-7a9b59eb-b0bf-4df6-bbc0-c59624df31cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09b552743cc2456791cd43550bcab466'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:34.353547 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-7a9b59eb-b0bf-4df6-bbc0-c59624df31cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09b552743cc2456791cd43550bcab466'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:34.361774 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-7a9b59eb-b0bf-4df6-bbc0-c59624df31cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09b552743cc2456791cd43550bcab466'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:34.366619 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:34.374673 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-7a9b59eb-b0bf-4df6-bbc0-c59624df31cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09b552743cc2456791cd43550bcab466'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:34.380883 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1271/2552] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:02:33 2022] DELETE /identity/v3/users/09b552743cc2456791cd43550bcab466 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:34.385733 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409c78b-b086-4e11-8a06-8d9a3db2d681 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:34.386217 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409c78b-b086-4e11-8a06-8d9a3db2d681 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:34.386325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a409c78b-b086-4e11-8a06-8d9a3db2d681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:34.405718 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-a409c78b-b086-4e11-8a06-8d9a3db2d681 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:34.407439 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-a409c78b-b086-4e11-8a06-8d9a3db2d681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:34.476466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1282/2553] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:34 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:34.486957 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b59eb-b0bf-4df6-bbc0-c59624df31cb admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:34.488632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-7a9b59eb-b0bf-4df6-bbc0-c59624df31cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:34.556224 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9b59eb-b0bf-4df6-bbc0-c59624df31cb admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:34.571847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76baf3-a0ca-40ec-818a-7c1b6453f8f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:34.571847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76baf3-a0ca-40ec-818a-7c1b6453f8f3 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:34.571847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76baf3-a0ca-40ec-818a-7c1b6453f8f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:34.571847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b76baf3-a0ca-40ec-818a-7c1b6453f8f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:34.572245 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b76baf3-a0ca-40ec-818a-7c1b6453f8f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:34.572245 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b76baf3-a0ca-40ec-818a-7c1b6453f8f3 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:34.581309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1272/2554] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:34 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:02:35.010841 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409c78b-b086-4e11-8a06-8d9a3db2d681 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:35.012980 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-a409c78b-b086-4e11-8a06-8d9a3db2d681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:35.031799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a409c78b-b086-4e11-8a06-8d9a3db2d681 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:35.038377 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ec30c-bd80-4637-8e20-02e0994dfe71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:35.039101 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ec30c-bd80-4637-8e20-02e0994dfe71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:35.039397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ec30c-bd80-4637-8e20-02e0994dfe71 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:35.039760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ec30c-bd80-4637-8e20-02e0994dfe71 admin admin] PATH_INFO: `/v3/projects/2548330e0145481ca7157bd2d138e902` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:35.040924 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ec30c-bd80-4637-8e20-02e0994dfe71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2548330e0145481ca7157bd2d138e902)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:35.042824 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ec30c-bd80-4637-8e20-02e0994dfe71 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:35.166681 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-a76ec30c-bd80-4637-8e20-02e0994dfe71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2548330e0145481ca7157bd2d138e902', 'request_id': 'req-a76ec30c-bd80-4637-8e20-02e0994dfe71'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:35.273689 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-a76ec30c-bd80-4637-8e20-02e0994dfe71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2548330e0145481ca7157bd2d138e902', 'request_id': 'req-a76ec30c-bd80-4637-8e20-02e0994dfe71'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:35.280131 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-a76ec30c-bd80-4637-8e20-02e0994dfe71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2548330e0145481ca7157bd2d138e902', 'request_id': 'req-a76ec30c-bd80-4637-8e20-02e0994dfe71'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:35.292678 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-a76ec30c-bd80-4637-8e20-02e0994dfe71 admin admin] The token cache is being invalidate because project 2548330e0145481ca7157bd2d138e902 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:02:35.292966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-a76ec30c-bd80-4637-8e20-02e0994dfe71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:35.293621 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-a76ec30c-bd80-4637-8e20-02e0994dfe71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:35.295391 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1283/2555] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:02:35 2022] DELETE /identity/v3/projects/2548330e0145481ca7157bd2d138e902 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:35.300828 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76baf3-a0ca-40ec-818a-7c1b6453f8f3 service neutron] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:35.310847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b76baf3-a0ca-40ec-818a-7c1b6453f8f3 service neutron] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:35.370025 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd627a4-1896-44e5-9c36-7327746b7ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:35.370721 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd627a4-1896-44e5-9c36-7327746b7ac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:35.370999 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd627a4-1896-44e5-9c36-7327746b7ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:35.371302 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd627a4-1896-44e5-9c36-7327746b7ac4 admin admin] PATH_INFO: `/v3/users/ca718594ffea4b51b143a91ddf7961cc` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:35.372495 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd627a4-1896-44e5-9c36-7327746b7ac4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca718594ffea4b51b143a91ddf7961cc)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:35.374409 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd627a4-1896-44e5-9c36-7327746b7ac4 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:35.601927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-bdd627a4-1896-44e5-9c36-7327746b7ac4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca718594ffea4b51b143a91ddf7961cc'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:35.602820 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-bdd627a4-1896-44e5-9c36-7327746b7ac4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca718594ffea4b51b143a91ddf7961cc'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:35.696585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-bdd627a4-1896-44e5-9c36-7327746b7ac4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca718594ffea4b51b143a91ddf7961cc'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:35.704173 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-bdd627a4-1896-44e5-9c36-7327746b7ac4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca718594ffea4b51b143a91ddf7961cc'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:35.716843 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-bdd627a4-1896-44e5-9c36-7327746b7ac4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca718594ffea4b51b143a91ddf7961cc'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:35.723748 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1273/2556] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:02:35 2022] DELETE /identity/v3/users/ca718594ffea4b51b143a91ddf7961cc => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:02:36.190250 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ec30c-bd80-4637-8e20-02e0994dfe71 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:36.192081 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-a76ec30c-bd80-4637-8e20-02e0994dfe71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:36.205537 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ec30c-bd80-4637-8e20-02e0994dfe71 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:36.256833 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:36.269929 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63d3a55-0500-444f-a771-cd71f7fa7b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:36.275898 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d3a55-0500-444f-a771-cd71f7fa7b98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:36.275898 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d3a55-0500-444f-a771-cd71f7fa7b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:36.275898 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63d3a55-0500-444f-a771-cd71f7fa7b98 admin admin] PATH_INFO: `/v3/projects/da7d85e2aad04693bf45a468c6cb113f` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:36.275898 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63d3a55-0500-444f-a771-cd71f7fa7b98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da7d85e2aad04693bf45a468c6cb113f)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:36.275898 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63d3a55-0500-444f-a771-cd71f7fa7b98 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:36.431216 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-e63d3a55-0500-444f-a771-cd71f7fa7b98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da7d85e2aad04693bf45a468c6cb113f', 'request_id': 'req-e63d3a55-0500-444f-a771-cd71f7fa7b98'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:36.519588 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-e63d3a55-0500-444f-a771-cd71f7fa7b98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da7d85e2aad04693bf45a468c6cb113f', 'request_id': 'req-e63d3a55-0500-444f-a771-cd71f7fa7b98'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:36.525947 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-e63d3a55-0500-444f-a771-cd71f7fa7b98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da7d85e2aad04693bf45a468c6cb113f', 'request_id': 'req-e63d3a55-0500-444f-a771-cd71f7fa7b98'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:36.537872 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-e63d3a55-0500-444f-a771-cd71f7fa7b98 admin admin] The token cache is being invalidate because project da7d85e2aad04693bf45a468c6cb113f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:02:36.538163 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-e63d3a55-0500-444f-a771-cd71f7fa7b98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:36.538902 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-e63d3a55-0500-444f-a771-cd71f7fa7b98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:36.540497 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1284/2557] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:02:36 2022] DELETE /identity/v3/projects/da7d85e2aad04693bf45a468c6cb113f => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:02:36.545422 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43d994-dca1-492d-9433-b1fd9054be10 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:36.545885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43d994-dca1-492d-9433-b1fd9054be10 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:36.546004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43d994-dca1-492d-9433-b1fd9054be10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:36.560514 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:36.566572 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-5f43d994-dca1-492d-9433-b1fd9054be10 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:36.567863 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-5f43d994-dca1-492d-9433-b1fd9054be10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:36.648718 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1274/2558] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:02:36.650157 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d4f66-c046-4edf-8e1b-7b92673c0573 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:36.650516 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d4f66-c046-4edf-8e1b-7b92673c0573 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:36.650897 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906d4f66-c046-4edf-8e1b-7b92673c0573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:36.673635 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-906d4f66-c046-4edf-8e1b-7b92673c0573 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:36.675815 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-906d4f66-c046-4edf-8e1b-7b92673c0573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:36.684065 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1285/2559] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:36 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:36.688698 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43d994-dca1-492d-9433-b1fd9054be10 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:36.698564 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43d994-dca1-492d-9433-b1fd9054be10 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:36.703895 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8134d-be8f-46f3-a852-627b31ae7ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:36.704486 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8134d-be8f-46f3-a852-627b31ae7ff8 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:36.704983 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8134d-be8f-46f3-a852-627b31ae7ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:36.705942 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a8134d-be8f-46f3-a852-627b31ae7ff8 admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:36.706758 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8134d-be8f-46f3-a852-627b31ae7ff8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:02:36.707087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8134d-be8f-46f3-a852-627b31ae7ff8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:36.708709 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a8134d-be8f-46f3-a852-627b31ae7ff8 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:36.720424 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1275/2560] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 11:02:36 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:02:36.725784 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d4f66-c046-4edf-8e1b-7b92673c0573 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:36.735366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906d4f66-c046-4edf-8e1b-7b92673c0573 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:36.741069 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ecbb44-b4c6-449e-b42f-b6f3c50175ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:36.741686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ecbb44-b4c6-449e-b42f-b6f3c50175ad admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:36.751583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ecbb44-b4c6-449e-b42f-b6f3c50175ad admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:36.751583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ecbb44-b4c6-449e-b42f-b6f3c50175ad admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:36.751583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ecbb44-b4c6-449e-b42f-b6f3c50175ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:36.751583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ecbb44-b4c6-449e-b42f-b6f3c50175ad admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:36.873873 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1286/2561] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:02:36 2022] POST /identity/v3/projects => generated 376 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:36.879413 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8134d-be8f-46f3-a852-627b31ae7ff8 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:36.889737 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a8134d-be8f-46f3-a852-627b31ae7ff8 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:36.953606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b1bd9-c23b-49ed-be48-3b2feca1b1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:36.954230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b1bd9-c23b-49ed-be48-3b2feca1b1cf admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:36.954640 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b1bd9-c23b-49ed-be48-3b2feca1b1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:36.954922 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789b1bd9-c23b-49ed-be48-3b2feca1b1cf admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:36.956134 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b1bd9-c23b-49ed-be48-3b2feca1b1cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:36.958198 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789b1bd9-c23b-49ed-be48-3b2feca1b1cf admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:37.062428 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1276/2562] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:02:36 2022] POST /identity/v3/users => generated 289 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:02:37.066573 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ecbb44-b4c6-449e-b42f-b6f3c50175ad admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:37.076775 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ecbb44-b4c6-449e-b42f-b6f3c50175ad admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:37.081929 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c309b-c122-4452-85e9-4085b6aff2aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:37.082530 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c309b-c122-4452-85e9-4085b6aff2aa admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:37.082809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c309b-c122-4452-85e9-4085b6aff2aa admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:37.083112 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3c309b-c122-4452-85e9-4085b6aff2aa admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:37.083804 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c309b-c122-4452-85e9-4085b6aff2aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:37.085480 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3c309b-c122-4452-85e9-4085b6aff2aa admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:37.098364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1287/2563] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:37 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:02:37.102632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b1bd9-c23b-49ed-be48-3b2feca1b1cf admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:37.116752 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789b1bd9-c23b-49ed-be48-3b2feca1b1cf admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:37.122988 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1231181e-1bde-4fca-bd92-a62e8428f00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:37.123805 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231181e-1bde-4fca-bd92-a62e8428f00f admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:37.124096 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231181e-1bde-4fca-bd92-a62e8428f00f admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:37.124624 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1231181e-1bde-4fca-bd92-a62e8428f00f admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:37.125309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1231181e-1bde-4fca-bd92-a62e8428f00f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:37.126914 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1231181e-1bde-4fca-bd92-a62e8428f00f admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:37.138933 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1277/2564] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:37 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:37.145038 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c309b-c122-4452-85e9-4085b6aff2aa admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:37.155607 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3c309b-c122-4452-85e9-4085b6aff2aa admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:37.161239 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520249ff-ee3a-4eeb-ad23-6ea50c950c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:37.161974 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520249ff-ee3a-4eeb-ad23-6ea50c950c08 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:37.162343 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520249ff-ee3a-4eeb-ad23-6ea50c950c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:37.162838 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520249ff-ee3a-4eeb-ad23-6ea50c950c08 admin admin] PATH_INFO: `/v3/projects/678992c01ee5434dbe3eb8faae0ef1ce/users/a0bc110516bd4b35812bebaea41ba019/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:37.182855 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520249ff-ee3a-4eeb-ad23-6ea50c950c08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=678992c01ee5434dbe3eb8faae0ef1ce, user_id=a0bc110516bd4b35812bebaea41ba019, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:37.184567 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520249ff-ee3a-4eeb-ad23-6ea50c950c08 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:37.314421 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1288/2565] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:37 2022] PUT /identity/v3/projects/678992c01ee5434dbe3eb8faae0ef1ce/users/a0bc110516bd4b35812bebaea41ba019/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:37.332026 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf61c30-bf05-42f8-b4eb-0aee8df84adf None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:37.332337 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf61c30-bf05-42f8-b4eb-0aee8df84adf None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:37.332617 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf61c30-bf05-42f8-b4eb-0aee8df84adf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:37.368143 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-bbf61c30-bf05-42f8-b4eb-0aee8df84adf None None] MFA Rules not processed for user `a0bc110516bd4b35812bebaea41ba019`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:37.369387 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-bbf61c30-bf05-42f8-b4eb-0aee8df84adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:37.444577 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1278/2566] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:37.465427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520249ff-ee3a-4eeb-ad23-6ea50c950c08 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:37.467478 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-520249ff-ee3a-4eeb-ad23-6ea50c950c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:37.529371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520249ff-ee3a-4eeb-ad23-6ea50c950c08 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:37.533120 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a786ad-c400-4cc0-85fd-fdaaac363611 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e358fd7c1b3c43d7ac89fb3562678266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:37.533755 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a786ad-c400-4cc0-85fd-fdaaac363611 service glance] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:37.534174 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a786ad-c400-4cc0-85fd-fdaaac363611 service glance] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:37.534482 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a786ad-c400-4cc0-85fd-fdaaac363611 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:37.542953 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a786ad-c400-4cc0-85fd-fdaaac363611 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:37.544414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a786ad-c400-4cc0-85fd-fdaaac363611 service glance] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:37.553762 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1289/2567] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:02:38.216631 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf61c30-bf05-42f8-b4eb-0aee8df84adf None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:38.218251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-bbf61c30-bf05-42f8-b4eb-0aee8df84adf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:38.224137 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:38.291387 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf61c30-bf05-42f8-b4eb-0aee8df84adf None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:38.296148 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ea8f4-fb4a-45d8-9460-7d86a23cd883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:38.296774 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ea8f4-fb4a-45d8-9460-7d86a23cd883 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:38.297155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ea8f4-fb4a-45d8-9460-7d86a23cd883 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:38.297409 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071ea8f4-fb4a-45d8-9460-7d86a23cd883 admin admin] PATH_INFO: `/v3/users/010b7165fd5b45e7877f5365db25abe8` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:38.298597 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ea8f4-fb4a-45d8-9460-7d86a23cd883 admin admin] RBAC: Authorizing `identity:delete_user(user_id=010b7165fd5b45e7877f5365db25abe8)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:38.301395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071ea8f4-fb4a-45d8-9460-7d86a23cd883 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:38.569287 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-071ea8f4-fb4a-45d8-9460-7d86a23cd883 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '010b7165fd5b45e7877f5365db25abe8'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:38.570081 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-071ea8f4-fb4a-45d8-9460-7d86a23cd883 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '010b7165fd5b45e7877f5365db25abe8'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:38.645249 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-071ea8f4-fb4a-45d8-9460-7d86a23cd883 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '010b7165fd5b45e7877f5365db25abe8'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:38.655115 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-071ea8f4-fb4a-45d8-9460-7d86a23cd883 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '010b7165fd5b45e7877f5365db25abe8'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:38.674190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-071ea8f4-fb4a-45d8-9460-7d86a23cd883 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '010b7165fd5b45e7877f5365db25abe8'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:38.683990 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1279/2568] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:02:38 2022] DELETE /identity/v3/users/010b7165fd5b45e7877f5365db25abe8 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:38.688596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552d82c-a35c-4914-b825-f0b42e6496a9 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:38.689100 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552d82c-a35c-4914-b825-f0b42e6496a9 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:38.689247 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a552d82c-a35c-4914-b825-f0b42e6496a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:38.709931 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-a552d82c-a35c-4914-b825-f0b42e6496a9 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:38.711138 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-a552d82c-a35c-4914-b825-f0b42e6496a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:38.723220 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:38.773368 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1290/2569] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:38 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:38.783180 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ea8f4-fb4a-45d8-9460-7d86a23cd883 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:38.784610 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-071ea8f4-fb4a-45d8-9460-7d86a23cd883 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:38.844410 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071ea8f4-fb4a-45d8-9460-7d86a23cd883 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:38.848333 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e34f40-0125-4ecd-984e-acbc2121a740 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:38.849078 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e34f40-0125-4ecd-984e-acbc2121a740 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:38.849381 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e34f40-0125-4ecd-984e-acbc2121a740 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:38.849660 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e34f40-0125-4ecd-984e-acbc2121a740 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:38.859220 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e34f40-0125-4ecd-984e-acbc2121a740 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:38.860723 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e34f40-0125-4ecd-984e-acbc2121a740 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:38.872501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1280/2570] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:38 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:02:39.387128 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a552d82c-a35c-4914-b825-f0b42e6496a9 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:39.388545 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-a552d82c-a35c-4914-b825-f0b42e6496a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:39.402066 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a552d82c-a35c-4914-b825-f0b42e6496a9 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:39.407674 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cc6407-5210-4a3c-ac6a-72739a1dc2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:39.408374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc6407-5210-4a3c-ac6a-72739a1dc2ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:39.408664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc6407-5210-4a3c-ac6a-72739a1dc2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:39.409008 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cc6407-5210-4a3c-ac6a-72739a1dc2ba admin admin] PATH_INFO: `/v3/projects/b397060521564f4cb278d81e397d1988` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:39.412514 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc6407-5210-4a3c-ac6a-72739a1dc2ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=b397060521564f4cb278d81e397d1988)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:39.415151 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cc6407-5210-4a3c-ac6a-72739a1dc2ba admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:39.621817 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-53cc6407-5210-4a3c-ac6a-72739a1dc2ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b397060521564f4cb278d81e397d1988', 'request_id': 'req-53cc6407-5210-4a3c-ac6a-72739a1dc2ba'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:39.725344 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-53cc6407-5210-4a3c-ac6a-72739a1dc2ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b397060521564f4cb278d81e397d1988', 'request_id': 'req-53cc6407-5210-4a3c-ac6a-72739a1dc2ba'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:39.733932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-53cc6407-5210-4a3c-ac6a-72739a1dc2ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b397060521564f4cb278d81e397d1988', 'request_id': 'req-53cc6407-5210-4a3c-ac6a-72739a1dc2ba'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:39.747865 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-53cc6407-5210-4a3c-ac6a-72739a1dc2ba admin admin] The token cache is being invalidate because project b397060521564f4cb278d81e397d1988 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:02:39.748148 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-53cc6407-5210-4a3c-ac6a-72739a1dc2ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:39.748854 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-53cc6407-5210-4a3c-ac6a-72739a1dc2ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:39.750450 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1291/2571] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:02:39 2022] DELETE /identity/v3/projects/b397060521564f4cb278d81e397d1988 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:39.755992 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61daab-9b1f-4673-b34f-35a4933f6d3f None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:39.756483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61daab-9b1f-4673-b34f-35a4933f6d3f None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:39.756584 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61daab-9b1f-4673-b34f-35a4933f6d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:39.777322 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-ae61daab-9b1f-4673-b34f-35a4933f6d3f None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:39.778657 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-ae61daab-9b1f-4673-b34f-35a4933f6d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:39.798337 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:39.837672 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1281/2572] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:39 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:39.846499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2bcc3-e7c7-4493-a465-28ce0ca791b7 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:39.846851 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2bcc3-e7c7-4493-a465-28ce0ca791b7 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:39.847319 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2bcc3-e7c7-4493-a465-28ce0ca791b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:39.867964 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-dfd2bcc3-e7c7-4493-a465-28ce0ca791b7 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:39.869170 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-dfd2bcc3-e7c7-4493-a465-28ce0ca791b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:39.877168 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1292/2573] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:39 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:39.881507 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae61daab-9b1f-4673-b34f-35a4933f6d3f None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:39.893165 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae61daab-9b1f-4673-b34f-35a4933f6d3f None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:39.896384 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265afc9d-7e22-4db1-a690-9339656f0581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:39.896994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265afc9d-7e22-4db1-a690-9339656f0581 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:39.897273 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265afc9d-7e22-4db1-a690-9339656f0581 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:39.897566 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265afc9d-7e22-4db1-a690-9339656f0581 admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:39.898494 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265afc9d-7e22-4db1-a690-9339656f0581 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:02:39.898798 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265afc9d-7e22-4db1-a690-9339656f0581 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:39.900421 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265afc9d-7e22-4db1-a690-9339656f0581 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:39.912613 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1282/2574] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 11:02:39 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 11:02:39.917083 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2bcc3-e7c7-4493-a465-28ce0ca791b7 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:39.928930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2bcc3-e7c7-4493-a465-28ce0ca791b7 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:39.934082 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb5725f-8200-4f1f-b69b-ed0786114068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:39.934690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5725f-8200-4f1f-b69b-ed0786114068 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:39.935020 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5725f-8200-4f1f-b69b-ed0786114068 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:39.935333 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb5725f-8200-4f1f-b69b-ed0786114068 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:39.935999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb5725f-8200-4f1f-b69b-ed0786114068 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:39.937962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb5725f-8200-4f1f-b69b-ed0786114068 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:40.038673 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1293/2575] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:02:39 2022] POST /identity/v3/projects => generated 376 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:02:40.041529 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265afc9d-7e22-4db1-a690-9339656f0581 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:40.053075 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265afc9d-7e22-4db1-a690-9339656f0581 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:40.115973 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a8f79-1390-454b-a815-5b30fe2a5fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:40.116594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a8f79-1390-454b-a815-5b30fe2a5fed admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:40.116880 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a8f79-1390-454b-a815-5b30fe2a5fed admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:40.117229 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a8f79-1390-454b-a815-5b30fe2a5fed admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:40.117940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a8f79-1390-454b-a815-5b30fe2a5fed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:40.119892 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2a8f79-1390-454b-a815-5b30fe2a5fed admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:40.260647 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1283/2576] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:02:40 2022] POST /identity/v3/users => generated 289 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:40.261837 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb5725f-8200-4f1f-b69b-ed0786114068 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:40.275618 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb5725f-8200-4f1f-b69b-ed0786114068 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:40.280725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c78f44-99a0-4dda-b343-807c18a29472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:40.281382 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c78f44-99a0-4dda-b343-807c18a29472 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:40.281659 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c78f44-99a0-4dda-b343-807c18a29472 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:40.281930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c78f44-99a0-4dda-b343-807c18a29472 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:40.282630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c78f44-99a0-4dda-b343-807c18a29472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:40.284270 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c78f44-99a0-4dda-b343-807c18a29472 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:40.294598 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1294/2577] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:40 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:02:40.298807 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a8f79-1390-454b-a815-5b30fe2a5fed admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:40.309319 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a8f79-1390-454b-a815-5b30fe2a5fed admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:40.314020 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a46fa4-9b16-4f4a-a274-3ca7a7badfde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:40.315282 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a46fa4-9b16-4f4a-a274-3ca7a7badfde admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:40.316770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a46fa4-9b16-4f4a-a274-3ca7a7badfde admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:40.316770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a46fa4-9b16-4f4a-a274-3ca7a7badfde admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:40.316770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a46fa4-9b16-4f4a-a274-3ca7a7badfde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:40.318228 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a46fa4-9b16-4f4a-a274-3ca7a7badfde admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:40.329558 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1284/2578] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:40 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:40.334610 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c78f44-99a0-4dda-b343-807c18a29472 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:40.344593 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c78f44-99a0-4dda-b343-807c18a29472 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:40.349164 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69db646-2955-44d0-8888-533f5266ccb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:40.349808 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69db646-2955-44d0-8888-533f5266ccb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:40.350395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69db646-2955-44d0-8888-533f5266ccb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:40.350673 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69db646-2955-44d0-8888-533f5266ccb4 admin admin] PATH_INFO: `/v3/projects/87f5c2685c34498e8a26bcf04cf49f06/users/a430e304b6514356a96f8be216d5efa2/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:40.365312 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69db646-2955-44d0-8888-533f5266ccb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87f5c2685c34498e8a26bcf04cf49f06, user_id=a430e304b6514356a96f8be216d5efa2, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:40.367992 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69db646-2955-44d0-8888-533f5266ccb4 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:40.468560 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1295/2579] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:40 2022] PUT /identity/v3/projects/87f5c2685c34498e8a26bcf04cf49f06/users/a430e304b6514356a96f8be216d5efa2/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:40.480135 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc378d-7cfd-4780-ab53-8ba3bd94c329 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:40.480469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc378d-7cfd-4780-ab53-8ba3bd94c329 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:40.480791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc378d-7cfd-4780-ab53-8ba3bd94c329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:40.510346 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-f1cc378d-7cfd-4780-ab53-8ba3bd94c329 None None] MFA Rules not processed for user `a430e304b6514356a96f8be216d5efa2`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:40.511643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-f1cc378d-7cfd-4780-ab53-8ba3bd94c329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:40.579406 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1285/2580] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:40.584810 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69db646-2955-44d0-8888-533f5266ccb4 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:40.625353 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:40.649094 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69db646-2955-44d0-8888-533f5266ccb4 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:40.653866 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74f2e6-b83f-41e4-88c7-945950ad3836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:40.659079 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74f2e6-b83f-41e4-88c7-945950ad3836 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:40.659079 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74f2e6-b83f-41e4-88c7-945950ad3836 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:40.659079 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74f2e6-b83f-41e4-88c7-945950ad3836 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:40.659079 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac74f2e6-b83f-41e4-88c7-945950ad3836 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:40.659079 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac74f2e6-b83f-41e4-88c7-945950ad3836 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:40.785173 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c469098-1529-4b6f-b95f-0bd69c9f501b None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:40.785764 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c469098-1529-4b6f-b95f-0bd69c9f501b None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:40.785764 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c469098-1529-4b6f-b95f-0bd69c9f501b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:40.806233 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-6c469098-1529-4b6f-b95f-0bd69c9f501b None None] MFA Rules not processed for user `914de37664914b4cb91e0ce2c3a0e1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:40.807040 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1296/2581] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:02:40 2022] POST /identity/v3/projects => generated 376 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:40.807717 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-6c469098-1529-4b6f-b95f-0bd69c9f501b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:40.812098 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74f2e6-b83f-41e4-88c7-945950ad3836 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:40.821614 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:40.871253 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1286/2582] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:40 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:40.878700 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74f2e6-b83f-41e4-88c7-945950ad3836 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:40.887548 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58b009-e9b0-4dcd-8ada-2ccfe4a4bb06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:40.887965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58b009-e9b0-4dcd-8ada-2ccfe4a4bb06 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:40.888243 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58b009-e9b0-4dcd-8ada-2ccfe4a4bb06 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:40.888722 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e58b009-e9b0-4dcd-8ada-2ccfe4a4bb06 admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:40.889387 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e58b009-e9b0-4dcd-8ada-2ccfe4a4bb06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:40.891367 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e58b009-e9b0-4dcd-8ada-2ccfe4a4bb06 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:40.985967 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1297/2583] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:02:40 2022] POST /identity/v3/users => generated 295 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:02:40.987120 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c469098-1529-4b6f-b95f-0bd69c9f501b None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:41.000481 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c469098-1529-4b6f-b95f-0bd69c9f501b None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:41.006699 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c5a94-8004-433b-9f90-a3c7f3e4ceeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:41.008525 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c5a94-8004-433b-9f90-a3c7f3e4ceeb admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:41.008525 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c5a94-8004-433b-9f90-a3c7f3e4ceeb admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:41.008525 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9c5a94-8004-433b-9f90-a3c7f3e4ceeb admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:41.008776 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c5a94-8004-433b-9f90-a3c7f3e4ceeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:41.010392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9c5a94-8004-433b-9f90-a3c7f3e4ceeb admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:41.022336 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1287/2584] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:40 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:02:41.026883 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58b009-e9b0-4dcd-8ada-2ccfe4a4bb06 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:41.041254 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e58b009-e9b0-4dcd-8ada-2ccfe4a4bb06 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:41.054576 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336dec8-2c39-4b6b-915d-24d99a79e1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:41.055589 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336dec8-2c39-4b6b-915d-24d99a79e1a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:41.056121 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336dec8-2c39-4b6b-915d-24d99a79e1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:41.056757 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336dec8-2c39-4b6b-915d-24d99a79e1a1 admin admin] PATH_INFO: `/v3/projects/a04242d51dbc4a9c866fbadcb0296a76/users/f769ab3bedd44a71bf2427101323054b/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:41.084743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336dec8-2c39-4b6b-915d-24d99a79e1a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a04242d51dbc4a9c866fbadcb0296a76, user_id=f769ab3bedd44a71bf2427101323054b, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:41.088070 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336dec8-2c39-4b6b-915d-24d99a79e1a1 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:41.174650 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1298/2585] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:41 2022] PUT /identity/v3/projects/a04242d51dbc4a9c866fbadcb0296a76/users/f769ab3bedd44a71bf2427101323054b/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:02:41.180383 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c5a94-8004-433b-9f90-a3c7f3e4ceeb admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:41.192365 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9c5a94-8004-433b-9f90-a3c7f3e4ceeb admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:41.276845 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca60c02-7bb2-4b26-9556-71816551dadd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:41.277479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca60c02-7bb2-4b26-9556-71816551dadd admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:41.277763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca60c02-7bb2-4b26-9556-71816551dadd admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:41.278490 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca60c02-7bb2-4b26-9556-71816551dadd admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:41.278815 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca60c02-7bb2-4b26-9556-71816551dadd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:41.281121 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca60c02-7bb2-4b26-9556-71816551dadd admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:41.293066 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1288/2586] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:41 2022] GET /identity/v3/roles => generated 1427 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:02:41.297374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336dec8-2c39-4b6b-915d-24d99a79e1a1 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:41.310029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336dec8-2c39-4b6b-915d-24d99a79e1a1 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:41.315147 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d1c55-c76c-42db-a288-52d12cf5c88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:41.316457 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d1c55-c76c-42db-a288-52d12cf5c88c admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:41.316771 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d1c55-c76c-42db-a288-52d12cf5c88c admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:41.317057 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d1c55-c76c-42db-a288-52d12cf5c88c admin admin] PATH_INFO: `/v3/projects/a04242d51dbc4a9c866fbadcb0296a76/users/f769ab3bedd44a71bf2427101323054b/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:41.319318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496d1c55-c76c-42db-a288-52d12cf5c88c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a04242d51dbc4a9c866fbadcb0296a76, user_id=f769ab3bedd44a71bf2427101323054b, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:41.326053 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496d1c55-c76c-42db-a288-52d12cf5c88c admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:41.338783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1299/2587] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:41 2022] PUT /identity/v3/projects/a04242d51dbc4a9c866fbadcb0296a76/users/f769ab3bedd44a71bf2427101323054b/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:41.349322 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81392880-3170-44c8-b26f-f95bbbe32d35 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:41.349639 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81392880-3170-44c8-b26f-f95bbbe32d35 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:41.349930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81392880-3170-44c8-b26f-f95bbbe32d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:41.380541 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-81392880-3170-44c8-b26f-f95bbbe32d35 None None] MFA Rules not processed for user `f769ab3bedd44a71bf2427101323054b`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:41.382055 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-81392880-3170-44c8-b26f-f95bbbe32d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:41.484215 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1289/2588] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:41 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:41.638572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d1c55-c76c-42db-a288-52d12cf5c88c admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:41.640878 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-496d1c55-c76c-42db-a288-52d12cf5c88c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:41.702018 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d1c55-c76c-42db-a288-52d12cf5c88c admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:41.708417 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0344a8-f19e-4f5a-b895-8df9e41e4591 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:41.709047 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0344a8-f19e-4f5a-b895-8df9e41e4591 service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:41.709328 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0344a8-f19e-4f5a-b895-8df9e41e4591 service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:41.709600 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0344a8-f19e-4f5a-b895-8df9e41e4591 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:41.741405 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:41.774384 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0344a8-f19e-4f5a-b895-8df9e41e4591 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:41.775971 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0344a8-f19e-4f5a-b895-8df9e41e4591 service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:41.786596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1300/2589] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:02:47.152618 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81392880-3170-44c8-b26f-f95bbbe32d35 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:47.236395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:47.265451 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81392880-3170-44c8-b26f-f95bbbe32d35 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:47.270567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdbec3f-e1f3-42bf-b2f6-443f51df722e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:47.271246 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdbec3f-e1f3-42bf-b2f6-443f51df722e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:47.271530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdbec3f-e1f3-42bf-b2f6-443f51df722e admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:47.271812 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdbec3f-e1f3-42bf-b2f6-443f51df722e admin admin] PATH_INFO: `/v3/users/a430e304b6514356a96f8be216d5efa2` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:47.273277 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdbec3f-e1f3-42bf-b2f6-443f51df722e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a430e304b6514356a96f8be216d5efa2)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:47.275234 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdbec3f-e1f3-42bf-b2f6-443f51df722e admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:47.531270 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-0cdbec3f-e1f3-42bf-b2f6-443f51df722e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a430e304b6514356a96f8be216d5efa2'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:47.531996 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-0cdbec3f-e1f3-42bf-b2f6-443f51df722e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a430e304b6514356a96f8be216d5efa2'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:47.658918 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-0cdbec3f-e1f3-42bf-b2f6-443f51df722e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a430e304b6514356a96f8be216d5efa2'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:47.668976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-0cdbec3f-e1f3-42bf-b2f6-443f51df722e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a430e304b6514356a96f8be216d5efa2'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:47.682220 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-0cdbec3f-e1f3-42bf-b2f6-443f51df722e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a430e304b6514356a96f8be216d5efa2'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:47.689761 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1290/2590] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:02:47 2022] DELETE /identity/v3/users/a430e304b6514356a96f8be216d5efa2 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:47.695076 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbec790-6712-4cb2-bcbf-632e90fd0505 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:47.695587 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbec790-6712-4cb2-bcbf-632e90fd0505 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:47.695726 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbec790-6712-4cb2-bcbf-632e90fd0505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:47.716930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-1bbec790-6712-4cb2-bcbf-632e90fd0505 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:47.718153 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-1bbec790-6712-4cb2-bcbf-632e90fd0505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:47.795724 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1301/2591] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:47 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:47.803326 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdbec3f-e1f3-42bf-b2f6-443f51df722e admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:47.804783 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-0cdbec3f-e1f3-42bf-b2f6-443f51df722e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:47.895793 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdbec3f-e1f3-42bf-b2f6-443f51df722e admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:47.899887 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027a6fe-bb5b-4a4a-b12b-1c24255cf433 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:47.900579 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027a6fe-bb5b-4a4a-b12b-1c24255cf433 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:47.900929 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027a6fe-bb5b-4a4a-b12b-1c24255cf433 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:47.901294 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027a6fe-bb5b-4a4a-b12b-1c24255cf433 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:47.921575 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4027a6fe-bb5b-4a4a-b12b-1c24255cf433 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:47.923310 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4027a6fe-bb5b-4a4a-b12b-1c24255cf433 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:47.936036 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1291/2592] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:47 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:02:48.589837 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbec790-6712-4cb2-bcbf-632e90fd0505 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:48.591645 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-1bbec790-6712-4cb2-bcbf-632e90fd0505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:48.687910 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbec790-6712-4cb2-bcbf-632e90fd0505 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:48.693322 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27408e-cb77-435a-883e-0cc2f5709a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:48.693955 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27408e-cb77-435a-883e-0cc2f5709a1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:48.694302 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27408e-cb77-435a-883e-0cc2f5709a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:48.694580 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac27408e-cb77-435a-883e-0cc2f5709a1f admin admin] PATH_INFO: `/v3/projects/87f5c2685c34498e8a26bcf04cf49f06` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:48.696283 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27408e-cb77-435a-883e-0cc2f5709a1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=87f5c2685c34498e8a26bcf04cf49f06)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:48.698565 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac27408e-cb77-435a-883e-0cc2f5709a1f admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:48.860123 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ac27408e-cb77-435a-883e-0cc2f5709a1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87f5c2685c34498e8a26bcf04cf49f06', 'request_id': 'req-ac27408e-cb77-435a-883e-0cc2f5709a1f'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:48.930145 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ac27408e-cb77-435a-883e-0cc2f5709a1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87f5c2685c34498e8a26bcf04cf49f06', 'request_id': 'req-ac27408e-cb77-435a-883e-0cc2f5709a1f'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:48.934575 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:48.936849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ac27408e-cb77-435a-883e-0cc2f5709a1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87f5c2685c34498e8a26bcf04cf49f06', 'request_id': 'req-ac27408e-cb77-435a-883e-0cc2f5709a1f'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:48.948567 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ac27408e-cb77-435a-883e-0cc2f5709a1f admin admin] The token cache is being invalidate because project 87f5c2685c34498e8a26bcf04cf49f06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:02:48.948850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ac27408e-cb77-435a-883e-0cc2f5709a1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:48.949505 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ac27408e-cb77-435a-883e-0cc2f5709a1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:48.951074 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1302/2593] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:02:48 2022] DELETE /identity/v3/projects/87f5c2685c34498e8a26bcf04cf49f06 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:02:48.955360 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027a6fe-bb5b-4a4a-b12b-1c24255cf433 service neutron] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:49.002708 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027a6fe-bb5b-4a4a-b12b-1c24255cf433 service neutron] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:49.025054 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:49.063145 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879f0a5c-0b9a-445d-aa18-a2c3f0348717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:49.063629 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f0a5c-0b9a-445d-aa18-a2c3f0348717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:49.063913 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f0a5c-0b9a-445d-aa18-a2c3f0348717 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:49.064193 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879f0a5c-0b9a-445d-aa18-a2c3f0348717 admin admin] PATH_INFO: `/v3/users/f769ab3bedd44a71bf2427101323054b` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:49.065484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879f0a5c-0b9a-445d-aa18-a2c3f0348717 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f769ab3bedd44a71bf2427101323054b)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:49.067590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879f0a5c-0b9a-445d-aa18-a2c3f0348717 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:49.294481 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-879f0a5c-0b9a-445d-aa18-a2c3f0348717 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f769ab3bedd44a71bf2427101323054b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:49.295954 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-879f0a5c-0b9a-445d-aa18-a2c3f0348717 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f769ab3bedd44a71bf2427101323054b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:49.387843 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-879f0a5c-0b9a-445d-aa18-a2c3f0348717 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f769ab3bedd44a71bf2427101323054b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:49.398741 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-879f0a5c-0b9a-445d-aa18-a2c3f0348717 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f769ab3bedd44a71bf2427101323054b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:49.436755 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-879f0a5c-0b9a-445d-aa18-a2c3f0348717 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f769ab3bedd44a71bf2427101323054b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:49.448796 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1292/2594] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:02:48 2022] DELETE /identity/v3/users/f769ab3bedd44a71bf2427101323054b => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:02:49.929648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27408e-cb77-435a-883e-0cc2f5709a1f admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:49.930327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-ac27408e-cb77-435a-883e-0cc2f5709a1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:49.944293 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac27408e-cb77-435a-883e-0cc2f5709a1f admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:50.005370 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a423d93-bc16-41ae-b9f2-8ac67b693c82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:50.005937 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a423d93-bc16-41ae-b9f2-8ac67b693c82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:50.006216 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a423d93-bc16-41ae-b9f2-8ac67b693c82 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:50.006490 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a423d93-bc16-41ae-b9f2-8ac67b693c82 admin admin] PATH_INFO: `/v3/projects/a04242d51dbc4a9c866fbadcb0296a76` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:50.007775 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a423d93-bc16-41ae-b9f2-8ac67b693c82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a04242d51dbc4a9c866fbadcb0296a76)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:50.009693 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a423d93-bc16-41ae-b9f2-8ac67b693c82 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:50.128536 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-4a423d93-bc16-41ae-b9f2-8ac67b693c82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a04242d51dbc4a9c866fbadcb0296a76', 'request_id': 'req-4a423d93-bc16-41ae-b9f2-8ac67b693c82'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:50.218346 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-4a423d93-bc16-41ae-b9f2-8ac67b693c82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a04242d51dbc4a9c866fbadcb0296a76', 'request_id': 'req-4a423d93-bc16-41ae-b9f2-8ac67b693c82'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:50.225921 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-4a423d93-bc16-41ae-b9f2-8ac67b693c82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a04242d51dbc4a9c866fbadcb0296a76', 'request_id': 'req-4a423d93-bc16-41ae-b9f2-8ac67b693c82'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:50.237414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-4a423d93-bc16-41ae-b9f2-8ac67b693c82 admin admin] The token cache is being invalidate because project a04242d51dbc4a9c866fbadcb0296a76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:02:50.237830 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-4a423d93-bc16-41ae-b9f2-8ac67b693c82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:50.238535 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-4a423d93-bc16-41ae-b9f2-8ac67b693c82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:50.240179 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1303/2595] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:02:49 2022] DELETE /identity/v3/projects/a04242d51dbc4a9c866fbadcb0296a76 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:50.245433 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66f290-6637-49cf-a411-ad6b41c57c42 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:50.245927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66f290-6637-49cf-a411-ad6b41c57c42 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:50.246029 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e66f290-6637-49cf-a411-ad6b41c57c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:50.308516 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-7e66f290-6637-49cf-a411-ad6b41c57c42 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:50.309793 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-7e66f290-6637-49cf-a411-ad6b41c57c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:50.372838 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1293/2596] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:50.383771 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56fd90-79ec-4a6d-844d-65b680656ac7 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:50.383771 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56fd90-79ec-4a6d-844d-65b680656ac7 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:50.383771 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c56fd90-79ec-4a6d-844d-65b680656ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:50.404912 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-1c56fd90-79ec-4a6d-844d-65b680656ac7 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:50.406141 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-1c56fd90-79ec-4a6d-844d-65b680656ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:50.414498 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1304/2597] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:50 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:50.419172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66f290-6637-49cf-a411-ad6b41c57c42 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:50.429808 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e66f290-6637-49cf-a411-ad6b41c57c42 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:50.435215 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae9aaa8-f8cc-4c10-a386-a3ab21c1946b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:50.438965 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9aaa8-f8cc-4c10-a386-a3ab21c1946b admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:50.438965 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9aaa8-f8cc-4c10-a386-a3ab21c1946b admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:50.438965 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae9aaa8-f8cc-4c10-a386-a3ab21c1946b admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:50.438965 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae9aaa8-f8cc-4c10-a386-a3ab21c1946b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:02:50.438965 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae9aaa8-f8cc-4c10-a386-a3ab21c1946b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:50.438965 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae9aaa8-f8cc-4c10-a386-a3ab21c1946b admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:50.448684 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:50.452135 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1294/2598] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 11:02:50 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:02:50.456733 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56fd90-79ec-4a6d-844d-65b680656ac7 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:50.467033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c56fd90-79ec-4a6d-844d-65b680656ac7 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:50.472037 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f284d-0e9f-4df2-95ce-a59cf7b32605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:50.472664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f284d-0e9f-4df2-95ce-a59cf7b32605 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:50.472944 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f284d-0e9f-4df2-95ce-a59cf7b32605 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:50.473219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055f284d-0e9f-4df2-95ce-a59cf7b32605 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:50.476649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f284d-0e9f-4df2-95ce-a59cf7b32605 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:50.476649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055f284d-0e9f-4df2-95ce-a59cf7b32605 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:50.636190 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1305/2599] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:02:50 2022] POST /identity/v3/projects => generated 374 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:02:50.640828 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae9aaa8-f8cc-4c10-a386-a3ab21c1946b admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:50.652574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae9aaa8-f8cc-4c10-a386-a3ab21c1946b admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:50.717665 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b08e7e-c390-4d62-b007-c558062936d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:50.718280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b08e7e-c390-4d62-b007-c558062936d3 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:50.718566 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b08e7e-c390-4d62-b007-c558062936d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:50.718842 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b08e7e-c390-4d62-b007-c558062936d3 admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:50.719580 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b08e7e-c390-4d62-b007-c558062936d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:50.722000 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b08e7e-c390-4d62-b007-c558062936d3 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:50.823110 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1295/2600] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:02:50 2022] POST /identity/v3/users => generated 288 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:50.827289 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f284d-0e9f-4df2-95ce-a59cf7b32605 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:50.838964 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055f284d-0e9f-4df2-95ce-a59cf7b32605 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:50.844466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a977057e-8540-4732-a0f8-5c697df007aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:50.845120 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a977057e-8540-4732-a0f8-5c697df007aa admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:50.845900 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a977057e-8540-4732-a0f8-5c697df007aa admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:50.845900 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a977057e-8540-4732-a0f8-5c697df007aa admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:50.847880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a977057e-8540-4732-a0f8-5c697df007aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:50.849841 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a977057e-8540-4732-a0f8-5c697df007aa admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:50.861907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1306/2601] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:50 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:02:50.866909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b08e7e-c390-4d62-b007-c558062936d3 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:50.879610 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b08e7e-c390-4d62-b007-c558062936d3 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:50.889959 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b2bdd-9d9c-4368-9358-aa9103a35d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:50.889959 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b2bdd-9d9c-4368-9358-aa9103a35d06 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:50.889959 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b2bdd-9d9c-4368-9358-aa9103a35d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:50.889959 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b2bdd-9d9c-4368-9358-aa9103a35d06 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:50.890530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62b2bdd-9d9c-4368-9358-aa9103a35d06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:50.891681 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62b2bdd-9d9c-4368-9358-aa9103a35d06 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:50.908447 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1296/2602] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:50 2022] GET /identity/v3/roles => generated 1427 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:50.916097 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a977057e-8540-4732-a0f8-5c697df007aa admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:50.939718 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a977057e-8540-4732-a0f8-5c697df007aa admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:50.948345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c6fd6f-c5fe-4b08-afc8-9adf6be6db2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:50.949053 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6fd6f-c5fe-4b08-afc8-9adf6be6db2d admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:50.949334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6fd6f-c5fe-4b08-afc8-9adf6be6db2d admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:50.949610 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c6fd6f-c5fe-4b08-afc8-9adf6be6db2d admin admin] PATH_INFO: `/v3/projects/4dccda1fb9a44c3d91811677bd6f36c3/users/d4ba1b5dac82438391a56b7cda25a363/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:50.966584 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6fd6f-c5fe-4b08-afc8-9adf6be6db2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4dccda1fb9a44c3d91811677bd6f36c3, user_id=d4ba1b5dac82438391a56b7cda25a363, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:50.969300 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c6fd6f-c5fe-4b08-afc8-9adf6be6db2d admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:51.002963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b2bdd-9d9c-4368-9358-aa9103a35d06 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:51.004731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-f62b2bdd-9d9c-4368-9358-aa9103a35d06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:51.017918 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b2bdd-9d9c-4368-9358-aa9103a35d06 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:51.022947 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c6563b-5032-4de7-b3ab-32415a140637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:51.023611 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6563b-5032-4de7-b3ab-32415a140637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:51.023892 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6563b-5032-4de7-b3ab-32415a140637 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:51.024164 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c6563b-5032-4de7-b3ab-32415a140637 admin admin] PATH_INFO: `/v3/users/7d100f35ab534f9caedf4dda6ca259e9` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:51.025334 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c6563b-5032-4de7-b3ab-32415a140637 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d100f35ab534f9caedf4dda6ca259e9)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:51.027291 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c6563b-5032-4de7-b3ab-32415a140637 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:51.037181 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1307/2603] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:50 2022] PUT /identity/v3/projects/4dccda1fb9a44c3d91811677bd6f36c3/users/d4ba1b5dac82438391a56b7cda25a363/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:51.042559 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6337ab-7f1c-4055-beb7-eb352f857a7e None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:51.046202 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6337ab-7f1c-4055-beb7-eb352f857a7e None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:51.046202 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6337ab-7f1c-4055-beb7-eb352f857a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:51.063352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-1c6337ab-7f1c-4055-beb7-eb352f857a7e None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:51.064574 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-1c6337ab-7f1c-4055-beb7-eb352f857a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:51.074139 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:51.124850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1308/2604] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:51 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:02:51.134540 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6337ab-7f1c-4055-beb7-eb352f857a7e None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:51.135924 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-1c6337ab-7f1c-4055-beb7-eb352f857a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:51.211932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6337ab-7f1c-4055-beb7-eb352f857a7e None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:51.212492 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-42c6563b-5032-4de7-b3ab-32415a140637 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d100f35ab534f9caedf4dda6ca259e9'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:51.213394 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-42c6563b-5032-4de7-b3ab-32415a140637 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d100f35ab534f9caedf4dda6ca259e9'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:51.216180 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e58fca-6abd-4e40-b989-7ad5274aa787 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:51.216795 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e58fca-6abd-4e40-b989-7ad5274aa787 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:51.217075 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e58fca-6abd-4e40-b989-7ad5274aa787 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:51.217351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e58fca-6abd-4e40-b989-7ad5274aa787 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:51.228350 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e58fca-6abd-4e40-b989-7ad5274aa787 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:51.229896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e58fca-6abd-4e40-b989-7ad5274aa787 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:51.239892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1309/2605] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:51 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:02:51.361299 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-42c6563b-5032-4de7-b3ab-32415a140637 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d100f35ab534f9caedf4dda6ca259e9'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:51.370181 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-42c6563b-5032-4de7-b3ab-32415a140637 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d100f35ab534f9caedf4dda6ca259e9'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:51.382646 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-42c6563b-5032-4de7-b3ab-32415a140637 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d100f35ab534f9caedf4dda6ca259e9'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:51.388735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1297/2606] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:02:51 2022] DELETE /identity/v3/users/7d100f35ab534f9caedf4dda6ca259e9 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:02:51.393304 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fbc83-3247-48c6-a7ed-2578833fd40c None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:51.394365 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fbc83-3247-48c6-a7ed-2578833fd40c None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:51.394528 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65fbc83-3247-48c6-a7ed-2578833fd40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:51.415177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-d65fbc83-3247-48c6-a7ed-2578833fd40c None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:51.416768 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-d65fbc83-3247-48c6-a7ed-2578833fd40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:51.494663 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1310/2607] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:51 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:51.504718 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c6563b-5032-4de7-b3ab-32415a140637 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:51.506122 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-42c6563b-5032-4de7-b3ab-32415a140637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:51.572086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c6563b-5032-4de7-b3ab-32415a140637 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:51.575590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c1e0ac-4e96-4b51-9baf-b9d8b8438dc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:51.576209 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1e0ac-4e96-4b51-9baf-b9d8b8438dc4 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:51.576516 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1e0ac-4e96-4b51-9baf-b9d8b8438dc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:51.576925 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c1e0ac-4e96-4b51-9baf-b9d8b8438dc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:51.584451 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:51.588963 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1e0ac-4e96-4b51-9baf-b9d8b8438dc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:51.590418 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c1e0ac-4e96-4b51-9baf-b9d8b8438dc4 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:51.601287 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1298/2608] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:51 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:02:52.248759 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65fbc83-3247-48c6-a7ed-2578833fd40c None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:52.250253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-d65fbc83-3247-48c6-a7ed-2578833fd40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:52.271869 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65fbc83-3247-48c6-a7ed-2578833fd40c None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:52.283600 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f74dc8-0300-4f51-b432-89d8df949106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:52.284981 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f74dc8-0300-4f51-b432-89d8df949106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:52.285263 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f74dc8-0300-4f51-b432-89d8df949106 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:52.285562 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f74dc8-0300-4f51-b432-89d8df949106 admin admin] PATH_INFO: `/v3/projects/38eae2393aa740d3af918c12d2ff1ec5` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:52.289027 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f74dc8-0300-4f51-b432-89d8df949106 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38eae2393aa740d3af918c12d2ff1ec5)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:52.289312 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f74dc8-0300-4f51-b432-89d8df949106 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:52.399152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:52.406177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-a8f74dc8-0300-4f51-b432-89d8df949106 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38eae2393aa740d3af918c12d2ff1ec5', 'request_id': 'req-a8f74dc8-0300-4f51-b432-89d8df949106'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:52.469425 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-a8f74dc8-0300-4f51-b432-89d8df949106 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38eae2393aa740d3af918c12d2ff1ec5', 'request_id': 'req-a8f74dc8-0300-4f51-b432-89d8df949106'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:52.479439 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-a8f74dc8-0300-4f51-b432-89d8df949106 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38eae2393aa740d3af918c12d2ff1ec5', 'request_id': 'req-a8f74dc8-0300-4f51-b432-89d8df949106'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:52.492647 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-a8f74dc8-0300-4f51-b432-89d8df949106 admin admin] The token cache is being invalidate because project 38eae2393aa740d3af918c12d2ff1ec5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:02:52.492901 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-a8f74dc8-0300-4f51-b432-89d8df949106 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:52.493575 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-a8f74dc8-0300-4f51-b432-89d8df949106 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:02:52.496284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1311/2609] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:02:52 2022] DELETE /identity/v3/projects/38eae2393aa740d3af918c12d2ff1ec5 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:52.501589 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05054c9a-f94f-48fc-9be8-a5c7528d1628 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:52.502123 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05054c9a-f94f-48fc-9be8-a5c7528d1628 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:52.502123 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05054c9a-f94f-48fc-9be8-a5c7528d1628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:52.543882 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-05054c9a-f94f-48fc-9be8-a5c7528d1628 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:52.543882 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-05054c9a-f94f-48fc-9be8-a5c7528d1628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:52.598795 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1299/2610] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:52.608066 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4df00-0c25-4702-905c-80e91f5fdda6 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:52.608352 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4df00-0c25-4702-905c-80e91f5fdda6 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:52.608976 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f4df00-0c25-4702-905c-80e91f5fdda6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:52.630652 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-96f4df00-0c25-4702-905c-80e91f5fdda6 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:52.631935 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-96f4df00-0c25-4702-905c-80e91f5fdda6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:52.640387 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1312/2611] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:52 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:52.645282 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05054c9a-f94f-48fc-9be8-a5c7528d1628 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:52.669744 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05054c9a-f94f-48fc-9be8-a5c7528d1628 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:52.674579 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e84f90-d8a2-4476-9315-17a5af60b378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:52.675242 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e84f90-d8a2-4476-9315-17a5af60b378 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:52.675526 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e84f90-d8a2-4476-9315-17a5af60b378 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:52.675850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e84f90-d8a2-4476-9315-17a5af60b378 admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:52.676553 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e84f90-d8a2-4476-9315-17a5af60b378 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:02:52.676858 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e84f90-d8a2-4476-9315-17a5af60b378 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:52.678455 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e84f90-d8a2-4476-9315-17a5af60b378 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:52.690253 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1300/2612] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 11:02:52 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:02:52.694895 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f4df00-0c25-4702-905c-80e91f5fdda6 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:52.704153 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f4df00-0c25-4702-905c-80e91f5fdda6 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:52.709374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033a1509-0bc3-4062-8a1c-20ed3a99e659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:52.710016 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a1509-0bc3-4062-8a1c-20ed3a99e659 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:52.710292 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a1509-0bc3-4062-8a1c-20ed3a99e659 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:52.710882 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033a1509-0bc3-4062-8a1c-20ed3a99e659 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:52.711270 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033a1509-0bc3-4062-8a1c-20ed3a99e659 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:52.713196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033a1509-0bc3-4062-8a1c-20ed3a99e659 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:52.835805 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1313/2613] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:02:52 2022] POST /identity/v3/projects => generated 378 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:52.839948 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e84f90-d8a2-4476-9315-17a5af60b378 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:52.851147 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e84f90-d8a2-4476-9315-17a5af60b378 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:52.880214 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ae0ce-7530-4c1e-8ec4-33c7e308bfa7 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:52.880625 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ae0ce-7530-4c1e-8ec4-33c7e308bfa7 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:52.881048 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9ae0ce-7530-4c1e-8ec4-33c7e308bfa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:52.915907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-cd9ae0ce-7530-4c1e-8ec4-33c7e308bfa7 None None] MFA Rules not processed for user `d4ba1b5dac82438391a56b7cda25a363`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:52.917470 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-cd9ae0ce-7530-4c1e-8ec4-33c7e308bfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:52.929735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f95df-fc34-489c-9f78-ad840a727efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:52.930369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f95df-fc34-489c-9f78-ad840a727efd admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:52.930744 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f95df-fc34-489c-9f78-ad840a727efd admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:52.930913 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f95df-fc34-489c-9f78-ad840a727efd admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:52.931612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f95df-fc34-489c-9f78-ad840a727efd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:52.933596 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f95df-fc34-489c-9f78-ad840a727efd admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:52.994699 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1314/2614] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:02:53.007691 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ae0ce-7530-4c1e-8ec4-33c7e308bfa7 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:53.009111 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-cd9ae0ce-7530-4c1e-8ec4-33c7e308bfa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:53.023087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1301/2615] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:02:52 2022] POST /identity/v3/users => generated 290 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:53.024427 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f95df-fc34-489c-9f78-ad840a727efd admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:53.034830 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f95df-fc34-489c-9f78-ad840a727efd admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:53.039629 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c449d-7609-43cd-9d91-d19c22e6b77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:53.040238 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c449d-7609-43cd-9d91-d19c22e6b77b admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:53.040520 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c449d-7609-43cd-9d91-d19c22e6b77b admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:53.040794 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578c449d-7609-43cd-9d91-d19c22e6b77b admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:53.041495 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578c449d-7609-43cd-9d91-d19c22e6b77b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:53.043138 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578c449d-7609-43cd-9d91-d19c22e6b77b admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:53.058503 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1302/2616] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:53 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:53.059731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c449d-7609-43cd-9d91-d19c22e6b77b admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:53.074116 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578c449d-7609-43cd-9d91-d19c22e6b77b admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:53.084433 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3771de-0a8b-4209-a03c-efe29be015d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:53.085166 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3771de-0a8b-4209-a03c-efe29be015d6 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:53.085528 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3771de-0a8b-4209-a03c-efe29be015d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:53.085810 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3771de-0a8b-4209-a03c-efe29be015d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:53.086518 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3771de-0a8b-4209-a03c-efe29be015d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:53.088259 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3771de-0a8b-4209-a03c-efe29be015d6 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:53.100712 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1303/2617] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:53 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:53.104991 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3771de-0a8b-4209-a03c-efe29be015d6 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:53.116487 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3771de-0a8b-4209-a03c-efe29be015d6 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:53.121507 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5275590-2d95-4a88-a3de-399dff178146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:53.122270 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5275590-2d95-4a88-a3de-399dff178146 admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:53.122606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5275590-2d95-4a88-a3de-399dff178146 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:53.122901 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5275590-2d95-4a88-a3de-399dff178146 admin admin] PATH_INFO: `/v3/projects/5d718235d1fd4a669b3cf06489a41334/users/d153ad318c904af3812b9778b323d2a0/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:53.139934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5275590-2d95-4a88-a3de-399dff178146 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d718235d1fd4a669b3cf06489a41334, user_id=d153ad318c904af3812b9778b323d2a0, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:53.145312 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5275590-2d95-4a88-a3de-399dff178146 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:53.203219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9ae0ce-7530-4c1e-8ec4-33c7e308bfa7 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:53.206768 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ff95c-4cf3-4f0a-8795-0d53851afc67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:53.208911 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ff95c-4cf3-4f0a-8795-0d53851afc67 service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:53.209188 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ff95c-4cf3-4f0a-8795-0d53851afc67 service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:53.209461 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322ff95c-4cf3-4f0a-8795-0d53851afc67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:53.217941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322ff95c-4cf3-4f0a-8795-0d53851afc67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:53.219412 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322ff95c-4cf3-4f0a-8795-0d53851afc67 service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:53.227754 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1304/2618] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:53 2022] PUT /identity/v3/projects/5d718235d1fd4a669b3cf06489a41334/users/d153ad318c904af3812b9778b323d2a0/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:02:53.229125 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1315/2619] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:02:53.238827 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a03f8a-2e84-43f0-8071-4eb4aa712402 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:53.239325 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a03f8a-2e84-43f0-8071-4eb4aa712402 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:53.239510 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a03f8a-2e84-43f0-8071-4eb4aa712402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:53.275925 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-57a03f8a-2e84-43f0-8071-4eb4aa712402 None None] MFA Rules not processed for user `d153ad318c904af3812b9778b323d2a0`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:53.276981 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-57a03f8a-2e84-43f0-8071-4eb4aa712402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:53.290303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:53.365229 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1305/2620] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:53 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:53.374455 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ff95c-4cf3-4f0a-8795-0d53851afc67 service cinder] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:53.391776 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322ff95c-4cf3-4f0a-8795-0d53851afc67 service cinder] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:53.432778 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:53.468202 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac8cd76-dfcc-49d1-8239-7fce2831a649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:53.473736 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8cd76-dfcc-49d1-8239-7fce2831a649 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:53.473736 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8cd76-dfcc-49d1-8239-7fce2831a649 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:53.473736 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac8cd76-dfcc-49d1-8239-7fce2831a649 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:53.473736 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac8cd76-dfcc-49d1-8239-7fce2831a649 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:53.473736 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac8cd76-dfcc-49d1-8239-7fce2831a649 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:53.641251 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1316/2621] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:02:53 2022] POST /identity/v3/projects => generated 374 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:53.642282 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a03f8a-2e84-43f0-8071-4eb4aa712402 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:53.653484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a03f8a-2e84-43f0-8071-4eb4aa712402 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:53.715169 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31690916-c0b1-4a6e-8b03-cf169c50c8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:53.715869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31690916-c0b1-4a6e-8b03-cf169c50c8ee admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:53.716157 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31690916-c0b1-4a6e-8b03-cf169c50c8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:53.716431 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31690916-c0b1-4a6e-8b03-cf169c50c8ee admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:53.717247 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31690916-c0b1-4a6e-8b03-cf169c50c8ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:53.719252 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31690916-c0b1-4a6e-8b03-cf169c50c8ee admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:53.851060 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1306/2622] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:02:53 2022] POST /identity/v3/users => generated 288 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:02:53.855015 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac8cd76-dfcc-49d1-8239-7fce2831a649 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:53.866017 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac8cd76-dfcc-49d1-8239-7fce2831a649 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:53.870461 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0456b-1fae-4402-99da-9bcc5b9de179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:53.871141 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0456b-1fae-4402-99da-9bcc5b9de179 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:53.871438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0456b-1fae-4402-99da-9bcc5b9de179 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:53.871726 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe0456b-1fae-4402-99da-9bcc5b9de179 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:53.872428 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe0456b-1fae-4402-99da-9bcc5b9de179 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:53.874022 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe0456b-1fae-4402-99da-9bcc5b9de179 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:53.884910 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1317/2623] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:53 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:53.889065 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31690916-c0b1-4a6e-8b03-cf169c50c8ee admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:53.900136 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31690916-c0b1-4a6e-8b03-cf169c50c8ee admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:53.904748 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24cce45-e7c6-4f37-b708-9011b0b7e9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:53.905362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24cce45-e7c6-4f37-b708-9011b0b7e9b1 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:53.905683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24cce45-e7c6-4f37-b708-9011b0b7e9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:53.911187 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24cce45-e7c6-4f37-b708-9011b0b7e9b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:53.911187 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24cce45-e7c6-4f37-b708-9011b0b7e9b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:53.911187 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24cce45-e7c6-4f37-b708-9011b0b7e9b1 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:53.919444 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1307/2624] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:53 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:53.923663 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0456b-1fae-4402-99da-9bcc5b9de179 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:53.935161 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe0456b-1fae-4402-99da-9bcc5b9de179 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:53.939665 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae90554-8df5-4246-b30d-67b06f8f847b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:53.940310 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae90554-8df5-4246-b30d-67b06f8f847b admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:53.940730 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae90554-8df5-4246-b30d-67b06f8f847b admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:53.940920 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae90554-8df5-4246-b30d-67b06f8f847b admin admin] PATH_INFO: `/v3/projects/2af85b5883ea4800a6f7b910a4168a18/users/4d1b065848924506a42e573f049a5422/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:53.955797 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae90554-8df5-4246-b30d-67b06f8f847b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2af85b5883ea4800a6f7b910a4168a18, user_id=4d1b065848924506a42e573f049a5422, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:53.958649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae90554-8df5-4246-b30d-67b06f8f847b admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:54.085700 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1318/2625] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:53 2022] PUT /identity/v3/projects/2af85b5883ea4800a6f7b910a4168a18/users/4d1b065848924506a42e573f049a5422/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:02:54.097369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8a14f-3567-4652-870c-b4e866033fc1 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:54.098339 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8a14f-3567-4652-870c-b4e866033fc1 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:54.098620 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c8a14f-3567-4652-870c-b4e866033fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:54.139871 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-b4c8a14f-3567-4652-870c-b4e866033fc1 None None] MFA Rules not processed for user `4d1b065848924506a42e573f049a5422`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:54.141185 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-b4c8a14f-3567-4652-870c-b4e866033fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:54.210134 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1308/2626] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:54.214661 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae90554-8df5-4246-b30d-67b06f8f847b admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:54.279131 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae90554-8df5-4246-b30d-67b06f8f847b admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:54.283871 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a21e7e-6174-415b-8d1e-9a5234128821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:54.284510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a21e7e-6174-415b-8d1e-9a5234128821 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:54.284806 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a21e7e-6174-415b-8d1e-9a5234128821 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:54.285783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a21e7e-6174-415b-8d1e-9a5234128821 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:54.286550 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a21e7e-6174-415b-8d1e-9a5234128821 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:54.288523 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a21e7e-6174-415b-8d1e-9a5234128821 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:54.386903 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1319/2627] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:02:54 2022] POST /identity/v3/projects => generated 376 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:54.391255 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8a14f-3567-4652-870c-b4e866033fc1 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:54.402228 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c8a14f-3567-4652-870c-b4e866033fc1 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:54.420731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:54.461007 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6bc486-8d91-403a-8f22-b88fe23bf6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:54.461666 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bc486-8d91-403a-8f22-b88fe23bf6ba admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:54.461949 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bc486-8d91-403a-8f22-b88fe23bf6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:54.462274 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6bc486-8d91-403a-8f22-b88fe23bf6ba admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:54.463126 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6bc486-8d91-403a-8f22-b88fe23bf6ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:54.465108 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6bc486-8d91-403a-8f22-b88fe23bf6ba admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:54.613850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1309/2628] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:02:54 2022] POST /identity/v3/users => generated 295 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:02:54.630952 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a21e7e-6174-415b-8d1e-9a5234128821 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:54.641198 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a21e7e-6174-415b-8d1e-9a5234128821 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:54.645982 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b7df9-17f1-4efc-9a3d-f607a435574a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:54.646613 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b7df9-17f1-4efc-9a3d-f607a435574a admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:54.646884 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b7df9-17f1-4efc-9a3d-f607a435574a admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:54.647203 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b7df9-17f1-4efc-9a3d-f607a435574a admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:54.648152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b7df9-17f1-4efc-9a3d-f607a435574a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:54.649728 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b7df9-17f1-4efc-9a3d-f607a435574a admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:54.660549 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1320/2629] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:54 2022] GET /identity/v3/roles => generated 1427 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:54.664721 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6bc486-8d91-403a-8f22-b88fe23bf6ba admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:54.674674 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6bc486-8d91-403a-8f22-b88fe23bf6ba admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:54.679855 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ccf48-97b8-4dbe-8847-acae85a36a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:54.680508 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ccf48-97b8-4dbe-8847-acae85a36a57 admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:54.680790 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ccf48-97b8-4dbe-8847-acae85a36a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:54.681060 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42ccf48-97b8-4dbe-8847-acae85a36a57 admin admin] PATH_INFO: `/v3/projects/79efeec1123e49248c8ceee2f8c82919/users/a6af19cab706481ba6c95ec4559ec72c/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:54.695709 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42ccf48-97b8-4dbe-8847-acae85a36a57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79efeec1123e49248c8ceee2f8c82919, user_id=a6af19cab706481ba6c95ec4559ec72c, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:54.698347 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42ccf48-97b8-4dbe-8847-acae85a36a57 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:54.778770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1310/2630] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:54 2022] PUT /identity/v3/projects/79efeec1123e49248c8ceee2f8c82919/users/a6af19cab706481ba6c95ec4559ec72c/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:54.782907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b7df9-17f1-4efc-9a3d-f607a435574a admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:54.793964 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b7df9-17f1-4efc-9a3d-f607a435574a admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:54.844928 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:54.852273 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302f3f1b-d282-427b-9345-4864c70f4883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:54.857521 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f3f1b-d282-427b-9345-4864c70f4883 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:54.857521 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f3f1b-d282-427b-9345-4864c70f4883 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:54.857521 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302f3f1b-d282-427b-9345-4864c70f4883 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:54.857521 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f3f1b-d282-427b-9345-4864c70f4883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:54.857521 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302f3f1b-d282-427b-9345-4864c70f4883 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:54.866005 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1321/2631] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:02:54 2022] GET /identity/v3/roles => generated 1427 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:02:54.870421 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ccf48-97b8-4dbe-8847-acae85a36a57 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:54.881267 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42ccf48-97b8-4dbe-8847-acae85a36a57 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:54.885764 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d61b7-f1cf-48ca-a3b2-26f19b2100af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:54.886444 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d61b7-f1cf-48ca-a3b2-26f19b2100af admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:54.886690 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d61b7-f1cf-48ca-a3b2-26f19b2100af admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:54.886960 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d61b7-f1cf-48ca-a3b2-26f19b2100af admin admin] PATH_INFO: `/v3/projects/79efeec1123e49248c8ceee2f8c82919/users/a6af19cab706481ba6c95ec4559ec72c/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:54.888943 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d61b7-f1cf-48ca-a3b2-26f19b2100af admin admin] RBAC: Authorizing `identity:create_grant(project_id=79efeec1123e49248c8ceee2f8c82919, user_id=a6af19cab706481ba6c95ec4559ec72c, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:54.891599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d61b7-f1cf-48ca-a3b2-26f19b2100af admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:54.902823 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1311/2632] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:02:54 2022] PUT /identity/v3/projects/79efeec1123e49248c8ceee2f8c82919/users/a6af19cab706481ba6c95ec4559ec72c/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:02:54.914158 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20aac31-61c3-4efb-83a2-c1d3b302dc53 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:54.914385 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20aac31-61c3-4efb-83a2-c1d3b302dc53 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:54.914666 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20aac31-61c3-4efb-83a2-c1d3b302dc53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:54.942290 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-d20aac31-61c3-4efb-83a2-c1d3b302dc53 None None] MFA Rules not processed for user `a6af19cab706481ba6c95ec4559ec72c`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:54.943544 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-d20aac31-61c3-4efb-83a2-c1d3b302dc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:55.010996 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1322/2633] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:54 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:55.022979 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d61b7-f1cf-48ca-a3b2-26f19b2100af admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:55.078134 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d61b7-f1cf-48ca-a3b2-26f19b2100af admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:55.082304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a923b10-fe42-40a5-b78f-d949ad83fcac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:55.082984 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a923b10-fe42-40a5-b78f-d949ad83fcac service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:55.083293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a923b10-fe42-40a5-b78f-d949ad83fcac service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:55.083569 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a923b10-fe42-40a5-b78f-d949ad83fcac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:55.141582 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a923b10-fe42-40a5-b78f-d949ad83fcac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:55.143067 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a923b10-fe42-40a5-b78f-d949ad83fcac service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:55.153076 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1312/2634] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:55 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:02:57.203397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20aac31-61c3-4efb-83a2-c1d3b302dc53 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:57.204804 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-d20aac31-61c3-4efb-83a2-c1d3b302dc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:57.216948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20aac31-61c3-4efb-83a2-c1d3b302dc53 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:57.220736 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab8a744-1734-49c3-9f52-5a43d5de66f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:57.221416 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8a744-1734-49c3-9f52-5a43d5de66f7 service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:57.221690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8a744-1734-49c3-9f52-5a43d5de66f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:57.221959 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab8a744-1734-49c3-9f52-5a43d5de66f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:57.282375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab8a744-1734-49c3-9f52-5a43d5de66f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:57.283831 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab8a744-1734-49c3-9f52-5a43d5de66f7 service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:57.293815 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1323/2635] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:02:57.348589 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57180844-d0e2-4abe-a1ec-3989e1b9b9ea None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:57.349132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57180844-d0e2-4abe-a1ec-3989e1b9b9ea None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:57.349132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57180844-d0e2-4abe-a1ec-3989e1b9b9ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:57.368409 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-57180844-d0e2-4abe-a1ec-3989e1b9b9ea None None] MFA Rules not processed for user `d4ba1b5dac82438391a56b7cda25a363`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:02:57.369624 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-57180844-d0e2-4abe-a1ec-3989e1b9b9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:57.395821 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:02:57.427380 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1313/2636] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:02:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:02:57.666394 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab8a744-1734-49c3-9f52-5a43d5de66f7 service cinder] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:57.667845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-2ab8a744-1734-49c3-9f52-5a43d5de66f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:57.729410 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab8a744-1734-49c3-9f52-5a43d5de66f7 service cinder] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:57.733050 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fcb4d8-bf91-4426-b8d2-553444dae6aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:57.733659 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fcb4d8-bf91-4426-b8d2-553444dae6aa service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:57.733937 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fcb4d8-bf91-4426-b8d2-553444dae6aa service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:57.734208 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fcb4d8-bf91-4426-b8d2-553444dae6aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:57.735983 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-38fcb4d8-bf91-4426-b8d2-553444dae6aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:57.746194 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fcb4d8-bf91-4426-b8d2-553444dae6aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:57.747658 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fcb4d8-bf91-4426-b8d2-553444dae6aa service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:57.755891 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1324/2637] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:57 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:02:58.065369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57180844-d0e2-4abe-a1ec-3989e1b9b9ea None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:58.067194 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-57180844-d0e2-4abe-a1ec-3989e1b9b9ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:58.080268 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57180844-d0e2-4abe-a1ec-3989e1b9b9ea None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:58.084522 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d3933-b7ce-4ec8-b1b1-e9a98aa59a4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:58.085147 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d3933-b7ce-4ec8-b1b1-e9a98aa59a4e service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:58.085610 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d3933-b7ce-4ec8-b1b1-e9a98aa59a4e service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:58.085899 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844d3933-b7ce-4ec8-b1b1-e9a98aa59a4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:58.087787 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-844d3933-b7ce-4ec8-b1b1-e9a98aa59a4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:02:58.164016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d3933-b7ce-4ec8-b1b1-e9a98aa59a4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:58.165542 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844d3933-b7ce-4ec8-b1b1-e9a98aa59a4e service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:58.175196 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1314/2638] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:02:59.323431 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fcb4d8-bf91-4426-b8d2-553444dae6aa service neutron] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:59.335836 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fcb4d8-bf91-4426-b8d2-553444dae6aa service neutron] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:02:59.340954 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d660cfc-5c5d-4cef-81b3-fc8f9336ded4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:02:59.341916 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d660cfc-5c5d-4cef-81b3-fc8f9336ded4 service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:02:59.342358 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d660cfc-5c5d-4cef-81b3-fc8f9336ded4 service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:02:59.342797 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d660cfc-5c5d-4cef-81b3-fc8f9336ded4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:02:59.362148 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d660cfc-5c5d-4cef-81b3-fc8f9336ded4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:02:59.363238 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d660cfc-5c5d-4cef-81b3-fc8f9336ded4 service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:02:59.375161 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1325/2639] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:59 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:02:59.933908 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d3933-b7ce-4ec8-b1b1-e9a98aa59a4e service neutron] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:02:59.936009 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-844d3933-b7ce-4ec8-b1b1-e9a98aa59a4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:00.078856 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844d3933-b7ce-4ec8-b1b1-e9a98aa59a4e service neutron] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:00.088060 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54238f5c-0933-4dc6-a488-4308b88bf440 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc4b962e2a742b08beef7187cc527e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:00.088751 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54238f5c-0933-4dc6-a488-4308b88bf440 service placement] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:00.089046 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54238f5c-0933-4dc6-a488-4308b88bf440 service placement] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:00.089374 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54238f5c-0933-4dc6-a488-4308b88bf440 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:00.096692 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-54238f5c-0933-4dc6-a488-4308b88bf440 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:00.112085 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54238f5c-0933-4dc6-a488-4308b88bf440 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:00.114488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54238f5c-0933-4dc6-a488-4308b88bf440 service placement] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:00.135142 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1315/2640] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:02:59 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:03:04.600367 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d660cfc-5c5d-4cef-81b3-fc8f9336ded4 service cinder] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:04.608443 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:04.667678 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54238f5c-0933-4dc6-a488-4308b88bf440 service placement] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:04.678196 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54238f5c-0933-4dc6-a488-4308b88bf440 service placement] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:04.682364 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfb5a09-f620-4d63-b75f-668f96feb2f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:04.682996 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb5a09-f620-4d63-b75f-668f96feb2f2 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:04.683306 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb5a09-f620-4d63-b75f-668f96feb2f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:04.683752 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfb5a09-f620-4d63-b75f-668f96feb2f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:04.685901 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-cbfb5a09-f620-4d63-b75f-668f96feb2f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:04.691505 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d660cfc-5c5d-4cef-81b3-fc8f9336ded4 service cinder] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:04.697497 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5b7152-18b9-4268-8868-22935550af9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:04.698439 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b7152-18b9-4268-8868-22935550af9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:04.698724 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b7152-18b9-4268-8868-22935550af9a admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:04.699607 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5b7152-18b9-4268-8868-22935550af9a admin admin] PATH_INFO: `/v3/users/d153ad318c904af3812b9778b323d2a0` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:04.701078 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5b7152-18b9-4268-8868-22935550af9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d153ad318c904af3812b9778b323d2a0)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:04.703812 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5b7152-18b9-4268-8868-22935550af9a admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:04.742886 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:04.769891 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfb5a09-f620-4d63-b75f-668f96feb2f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:04.772547 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbfb5a09-f620-4d63-b75f-668f96feb2f2 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:04.784618 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1316/2641] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:03:04 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:03:04.930039 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-bb5b7152-18b9-4268-8868-22935550af9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd153ad318c904af3812b9778b323d2a0'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:04.941624 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-bb5b7152-18b9-4268-8868-22935550af9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd153ad318c904af3812b9778b323d2a0'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:04.942649 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-bb5b7152-18b9-4268-8868-22935550af9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd153ad318c904af3812b9778b323d2a0'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:05.035985 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-bb5b7152-18b9-4268-8868-22935550af9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd153ad318c904af3812b9778b323d2a0'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:05.054149 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-bb5b7152-18b9-4268-8868-22935550af9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd153ad318c904af3812b9778b323d2a0'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:05.063711 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1326/2642] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:03:04 2022] DELETE /identity/v3/users/d153ad318c904af3812b9778b323d2a0 => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:05.065456 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d2f08-2f8b-4ebc-8c79-4eb3544f18de None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:05.065814 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d2f08-2f8b-4ebc-8c79-4eb3544f18de None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:05.066099 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215d2f08-2f8b-4ebc-8c79-4eb3544f18de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:05.088011 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-215d2f08-2f8b-4ebc-8c79-4eb3544f18de None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:05.089244 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-215d2f08-2f8b-4ebc-8c79-4eb3544f18de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:05.229024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1317/2643] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:03:05 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:03:05.248534 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5b7152-18b9-4268-8868-22935550af9a admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:05.255630 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-bb5b7152-18b9-4268-8868-22935550af9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:05.338473 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5b7152-18b9-4268-8868-22935550af9a admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:05.342956 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e541b4-6436-4ccd-8138-82c0995ec59c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:05.343641 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e541b4-6436-4ccd-8138-82c0995ec59c service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:05.344555 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e541b4-6436-4ccd-8138-82c0995ec59c service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:05.344836 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e541b4-6436-4ccd-8138-82c0995ec59c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:05.372603 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e541b4-6436-4ccd-8138-82c0995ec59c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:05.374526 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e541b4-6436-4ccd-8138-82c0995ec59c service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:05.390134 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1327/2644] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:03:05 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:03:06.110964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d2f08-2f8b-4ebc-8c79-4eb3544f18de None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:06.112437 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-215d2f08-2f8b-4ebc-8c79-4eb3544f18de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:06.131690 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215d2f08-2f8b-4ebc-8c79-4eb3544f18de None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:06.158448 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c009cf-fb5e-4e05-945a-198fe53b161b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:06.161213 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c009cf-fb5e-4e05-945a-198fe53b161b admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:06.164167 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c009cf-fb5e-4e05-945a-198fe53b161b admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:06.164167 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c009cf-fb5e-4e05-945a-198fe53b161b admin admin] PATH_INFO: `/v3/projects/5d718235d1fd4a669b3cf06489a41334` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:06.164167 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c009cf-fb5e-4e05-945a-198fe53b161b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d718235d1fd4a669b3cf06489a41334)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:06.166113 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c009cf-fb5e-4e05-945a-198fe53b161b admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:06.329331 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-08c009cf-fb5e-4e05-945a-198fe53b161b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d718235d1fd4a669b3cf06489a41334', 'request_id': 'req-08c009cf-fb5e-4e05-945a-198fe53b161b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:06.340277 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-08c009cf-fb5e-4e05-945a-198fe53b161b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d718235d1fd4a669b3cf06489a41334', 'request_id': 'req-08c009cf-fb5e-4e05-945a-198fe53b161b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:06.421645 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-08c009cf-fb5e-4e05-945a-198fe53b161b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d718235d1fd4a669b3cf06489a41334', 'request_id': 'req-08c009cf-fb5e-4e05-945a-198fe53b161b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:06.437394 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-08c009cf-fb5e-4e05-945a-198fe53b161b admin admin] The token cache is being invalidate because project 5d718235d1fd4a669b3cf06489a41334 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:03:06.437793 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-08c009cf-fb5e-4e05-945a-198fe53b161b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:06.438440 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-08c009cf-fb5e-4e05-945a-198fe53b161b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:06.440044 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1318/2645] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:03:06 2022] DELETE /identity/v3/projects/5d718235d1fd4a669b3cf06489a41334 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:03:06.445115 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e541b4-6436-4ccd-8138-82c0995ec59c service neutron] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:06.457987 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e541b4-6436-4ccd-8138-82c0995ec59c service neutron] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:06.486330 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:06.533234 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac538c8c-cbc8-4fe7-ad72-db3ce8cb8baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:06.533904 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac538c8c-cbc8-4fe7-ad72-db3ce8cb8baa admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:06.534178 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac538c8c-cbc8-4fe7-ad72-db3ce8cb8baa admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:06.534725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac538c8c-cbc8-4fe7-ad72-db3ce8cb8baa admin admin] PATH_INFO: `/v3/users/4d1b065848924506a42e573f049a5422` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:06.536152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac538c8c-cbc8-4fe7-ad72-db3ce8cb8baa admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d1b065848924506a42e573f049a5422)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:06.540785 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac538c8c-cbc8-4fe7-ad72-db3ce8cb8baa admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:06.828417 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ac538c8c-cbc8-4fe7-ad72-db3ce8cb8baa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d1b065848924506a42e573f049a5422'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:06.836235 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ac538c8c-cbc8-4fe7-ad72-db3ce8cb8baa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d1b065848924506a42e573f049a5422'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:06.836932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ac538c8c-cbc8-4fe7-ad72-db3ce8cb8baa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d1b065848924506a42e573f049a5422'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:06.911581 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c009cf-fb5e-4e05-945a-198fe53b161b admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:06.913345 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-08c009cf-fb5e-4e05-945a-198fe53b161b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:06.932727 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ac538c8c-cbc8-4fe7-ad72-db3ce8cb8baa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d1b065848924506a42e573f049a5422'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:06.969509 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ac538c8c-cbc8-4fe7-ad72-db3ce8cb8baa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d1b065848924506a42e573f049a5422'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:06.982984 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1328/2646] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:03:06 2022] DELETE /identity/v3/users/4d1b065848924506a42e573f049a5422 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:07.025196 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:07.052729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c009cf-fb5e-4e05-945a-198fe53b161b admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:07.130859 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56dfea-5ca2-4b28-ac18-4ed0a228725a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:07.131609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56dfea-5ca2-4b28-ac18-4ed0a228725a service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:07.131786 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56dfea-5ca2-4b28-ac18-4ed0a228725a service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:07.132088 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a56dfea-5ca2-4b28-ac18-4ed0a228725a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:07.133865 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-5a56dfea-5ca2-4b28-ac18-4ed0a228725a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:07.204212 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56dfea-5ca2-4b28-ac18-4ed0a228725a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:07.205856 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a56dfea-5ca2-4b28-ac18-4ed0a228725a service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:07.219577 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1319/2647] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:03:06 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:03:07.462072 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac538c8c-cbc8-4fe7-ad72-db3ce8cb8baa admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:07.463572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-ac538c8c-cbc8-4fe7-ad72-db3ce8cb8baa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:07.474472 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56dfea-5ca2-4b28-ac18-4ed0a228725a service neutron] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:07.476455 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-5a56dfea-5ca2-4b28-ac18-4ed0a228725a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:07.483355 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac538c8c-cbc8-4fe7-ad72-db3ce8cb8baa admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:07.494956 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30781a0-2fb8-480a-9bf8-c4aa73a30b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:07.494956 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30781a0-2fb8-480a-9bf8-c4aa73a30b5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:07.494956 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30781a0-2fb8-480a-9bf8-c4aa73a30b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:07.494956 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30781a0-2fb8-480a-9bf8-c4aa73a30b5d admin admin] PATH_INFO: `/v3/users/a0bc110516bd4b35812bebaea41ba019` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:07.495359 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30781a0-2fb8-480a-9bf8-c4aa73a30b5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0bc110516bd4b35812bebaea41ba019)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:07.496159 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30781a0-2fb8-480a-9bf8-c4aa73a30b5d admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:07.497447 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a56dfea-5ca2-4b28-ac18-4ed0a228725a service neutron] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:07.504544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05b6be-afde-41ce-bb6e-c883cea73143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:07.505544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05b6be-afde-41ce-bb6e-c883cea73143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:07.506221 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05b6be-afde-41ce-bb6e-c883cea73143 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:07.506718 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb05b6be-afde-41ce-bb6e-c883cea73143 admin admin] PATH_INFO: `/v3/projects/2af85b5883ea4800a6f7b910a4168a18` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:07.509453 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb05b6be-afde-41ce-bb6e-c883cea73143 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2af85b5883ea4800a6f7b910a4168a18)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:07.514212 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb05b6be-afde-41ce-bb6e-c883cea73143 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:07.774439 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-c30781a0-2fb8-480a-9bf8-c4aa73a30b5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0bc110516bd4b35812bebaea41ba019'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:07.777338 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-fb05b6be-afde-41ce-bb6e-c883cea73143 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2af85b5883ea4800a6f7b910a4168a18', 'request_id': 'req-fb05b6be-afde-41ce-bb6e-c883cea73143'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:07.794713 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-c30781a0-2fb8-480a-9bf8-c4aa73a30b5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0bc110516bd4b35812bebaea41ba019'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:07.795563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-c30781a0-2fb8-480a-9bf8-c4aa73a30b5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0bc110516bd4b35812bebaea41ba019'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:07.798915 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-fb05b6be-afde-41ce-bb6e-c883cea73143 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2af85b5883ea4800a6f7b910a4168a18', 'request_id': 'req-fb05b6be-afde-41ce-bb6e-c883cea73143'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:07.896151 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-c30781a0-2fb8-480a-9bf8-c4aa73a30b5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0bc110516bd4b35812bebaea41ba019'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:07.918231 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-c30781a0-2fb8-480a-9bf8-c4aa73a30b5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0bc110516bd4b35812bebaea41ba019'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:07.939366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1329/2648] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:03:07 2022] DELETE /identity/v3/users/a0bc110516bd4b35812bebaea41ba019 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:07.941066 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42607823-6852-4f98-98d0-4aec9b5d3f11 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:07.941473 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42607823-6852-4f98-98d0-4aec9b5d3f11 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:07.942035 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42607823-6852-4f98-98d0-4aec9b5d3f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:07.970146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-42607823-6852-4f98-98d0-4aec9b5d3f11 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:07.971400 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-42607823-6852-4f98-98d0-4aec9b5d3f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:07.988051 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-fb05b6be-afde-41ce-bb6e-c883cea73143 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2af85b5883ea4800a6f7b910a4168a18', 'request_id': 'req-fb05b6be-afde-41ce-bb6e-c883cea73143'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:08.006939 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-fb05b6be-afde-41ce-bb6e-c883cea73143 admin admin] The token cache is being invalidate because project 2af85b5883ea4800a6f7b910a4168a18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:03:08.010322 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-fb05b6be-afde-41ce-bb6e-c883cea73143 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:08.028071 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-fb05b6be-afde-41ce-bb6e-c883cea73143 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:08.039499 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1320/2649] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:03:07 2022] DELETE /identity/v3/projects/2af85b5883ea4800a6f7b910a4168a18 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:08.043566 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05b6be-afde-41ce-bb6e-c883cea73143 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:08.053310 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-fb05b6be-afde-41ce-bb6e-c883cea73143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:08.134004 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:08.158125 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1330/2650] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:03:07 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:08.195556 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42607823-6852-4f98-98d0-4aec9b5d3f11 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:08.197006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-42607823-6852-4f98-98d0-4aec9b5d3f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:08.226014 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb05b6be-afde-41ce-bb6e-c883cea73143 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:08.234251 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77549b0d-bdda-43c4-9b6c-9e5841d7a541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:08.236791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77549b0d-bdda-43c4-9b6c-9e5841d7a541 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:08.236791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77549b0d-bdda-43c4-9b6c-9e5841d7a541 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:08.236791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77549b0d-bdda-43c4-9b6c-9e5841d7a541 admin admin] PATH_INFO: `/v3/users/a6af19cab706481ba6c95ec4559ec72c` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:08.238456 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77549b0d-bdda-43c4-9b6c-9e5841d7a541 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6af19cab706481ba6c95ec4559ec72c)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:08.242697 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77549b0d-bdda-43c4-9b6c-9e5841d7a541 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:08.324502 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42607823-6852-4f98-98d0-4aec9b5d3f11 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:08.340273 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9f78e-174c-412b-8446-3f6b61e29c9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:08.341312 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9f78e-174c-412b-8446-3f6b61e29c9a service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:08.341312 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9f78e-174c-412b-8446-3f6b61e29c9a service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:08.341667 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee9f78e-174c-412b-8446-3f6b61e29c9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:08.356795 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee9f78e-174c-412b-8446-3f6b61e29c9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:08.358257 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee9f78e-174c-412b-8446-3f6b61e29c9a service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:08.370622 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1331/2651] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:03:08 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:03:08.544444 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-77549b0d-bdda-43c4-9b6c-9e5841d7a541 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6af19cab706481ba6c95ec4559ec72c'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:08.546253 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-77549b0d-bdda-43c4-9b6c-9e5841d7a541 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6af19cab706481ba6c95ec4559ec72c'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:08.637625 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:08.639670 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-77549b0d-bdda-43c4-9b6c-9e5841d7a541 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6af19cab706481ba6c95ec4559ec72c'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:08.654063 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-77549b0d-bdda-43c4-9b6c-9e5841d7a541 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6af19cab706481ba6c95ec4559ec72c'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:08.718866 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-77549b0d-bdda-43c4-9b6c-9e5841d7a541 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6af19cab706481ba6c95ec4559ec72c'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:08.733286 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1321/2652] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:03:08 2022] DELETE /identity/v3/users/a6af19cab706481ba6c95ec4559ec72c => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:09.844670 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9f78e-174c-412b-8446-3f6b61e29c9a service neutron] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:09.866292 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee9f78e-174c-412b-8446-3f6b61e29c9a service neutron] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:09.878515 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77549b0d-bdda-43c4-9b6c-9e5841d7a541 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:09.880280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-77549b0d-bdda-43c4-9b6c-9e5841d7a541 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:09.973945 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77549b0d-bdda-43c4-9b6c-9e5841d7a541 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:09.980371 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78117f99-bda0-4c99-8280-16a2d3d0f757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:09.981026 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78117f99-bda0-4c99-8280-16a2d3d0f757 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:09.981460 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78117f99-bda0-4c99-8280-16a2d3d0f757 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:09.981748 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78117f99-bda0-4c99-8280-16a2d3d0f757 admin admin] PATH_INFO: `/v3/projects/678992c01ee5434dbe3eb8faae0ef1ce` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:09.983093 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78117f99-bda0-4c99-8280-16a2d3d0f757 admin admin] RBAC: Authorizing `identity:delete_project(project_id=678992c01ee5434dbe3eb8faae0ef1ce)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:09.989642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78117f99-bda0-4c99-8280-16a2d3d0f757 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:09.989909 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f6339-9477-4774-b4ff-44a29e8da825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:09.989909 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f6339-9477-4774-b4ff-44a29e8da825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:09.989909 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f6339-9477-4774-b4ff-44a29e8da825 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:09.989909 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f6339-9477-4774-b4ff-44a29e8da825 admin admin] PATH_INFO: `/v3/projects/79efeec1123e49248c8ceee2f8c82919` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:09.990241 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f6339-9477-4774-b4ff-44a29e8da825 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79efeec1123e49248c8ceee2f8c82919)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:09.992162 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f6339-9477-4774-b4ff-44a29e8da825 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:10.147962 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-78117f99-bda0-4c99-8280-16a2d3d0f757 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '678992c01ee5434dbe3eb8faae0ef1ce', 'request_id': 'req-78117f99-bda0-4c99-8280-16a2d3d0f757'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:10.157167 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-a43f6339-9477-4774-b4ff-44a29e8da825 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79efeec1123e49248c8ceee2f8c82919', 'request_id': 'req-a43f6339-9477-4774-b4ff-44a29e8da825'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:10.158698 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-78117f99-bda0-4c99-8280-16a2d3d0f757 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '678992c01ee5434dbe3eb8faae0ef1ce', 'request_id': 'req-78117f99-bda0-4c99-8280-16a2d3d0f757'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:10.258292 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-a43f6339-9477-4774-b4ff-44a29e8da825 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79efeec1123e49248c8ceee2f8c82919', 'request_id': 'req-a43f6339-9477-4774-b4ff-44a29e8da825'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:10.265615 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-a43f6339-9477-4774-b4ff-44a29e8da825 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79efeec1123e49248c8ceee2f8c82919', 'request_id': 'req-a43f6339-9477-4774-b4ff-44a29e8da825'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:10.278502 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-a43f6339-9477-4774-b4ff-44a29e8da825 admin admin] The token cache is being invalidate because project 79efeec1123e49248c8ceee2f8c82919 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:03:10.278861 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-a43f6339-9477-4774-b4ff-44a29e8da825 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:10.280070 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-a43f6339-9477-4774-b4ff-44a29e8da825 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:10.281512 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1332/2653] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:03:09 2022] DELETE /identity/v3/projects/79efeec1123e49248c8ceee2f8c82919 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:10.289363 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ebb1c-b47f-4af4-a57f-28a7a6e0cb28 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:10.289657 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ebb1c-b47f-4af4-a57f-28a7a6e0cb28 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:10.289939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ebb1c-b47f-4af4-a57f-28a7a6e0cb28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:10.312655 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-3e6ebb1c-b47f-4af4-a57f-28a7a6e0cb28 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:10.313936 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-3e6ebb1c-b47f-4af4-a57f-28a7a6e0cb28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:10.379469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1333/2654] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:03:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:03:10.386184 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-78117f99-bda0-4c99-8280-16a2d3d0f757 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '678992c01ee5434dbe3eb8faae0ef1ce', 'request_id': 'req-78117f99-bda0-4c99-8280-16a2d3d0f757'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:10.388104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96253ed5-92e7-4897-9ff5-ab3ae9c8ec06 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:10.388104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96253ed5-92e7-4897-9ff5-ab3ae9c8ec06 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:10.388104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96253ed5-92e7-4897-9ff5-ab3ae9c8ec06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:10.408598 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-96253ed5-92e7-4897-9ff5-ab3ae9c8ec06 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:10.410440 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-96253ed5-92e7-4897-9ff5-ab3ae9c8ec06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:10.419820 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1334/2655] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:03:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:10.424361 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96253ed5-92e7-4897-9ff5-ab3ae9c8ec06 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:10.437454 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-78117f99-bda0-4c99-8280-16a2d3d0f757 admin admin] The token cache is being invalidate because project 678992c01ee5434dbe3eb8faae0ef1ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:03:10.438714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96253ed5-92e7-4897-9ff5-ab3ae9c8ec06 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:10.440823 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-78117f99-bda0-4c99-8280-16a2d3d0f757 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:10.444119 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-78117f99-bda0-4c99-8280-16a2d3d0f757 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:10.448742 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1322/2656] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:03:09 2022] DELETE /identity/v3/projects/678992c01ee5434dbe3eb8faae0ef1ce => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:03:10.452146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49d96d-0445-4019-95d7-9f8bc33a06ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:10.452146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49d96d-0445-4019-95d7-9f8bc33a06ba admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:10.452146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49d96d-0445-4019-95d7-9f8bc33a06ba admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:10.452146 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe49d96d-0445-4019-95d7-9f8bc33a06ba admin admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:10.453486 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe49d96d-0445-4019-95d7-9f8bc33a06ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:03:10.453740 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe49d96d-0445-4019-95d7-9f8bc33a06ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:10.455199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0667be-2bb9-49de-ba8e-f11080f05e86 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:10.457719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe49d96d-0445-4019-95d7-9f8bc33a06ba admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:10.458265 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0667be-2bb9-49de-ba8e-f11080f05e86 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:10.458710 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0667be-2bb9-49de-ba8e-f11080f05e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:10.484395 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1335/2657] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 11:03:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 11:03:10.485529 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49d96d-0445-4019-95d7-9f8bc33a06ba admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:10.487968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-fe49d96d-0445-4019-95d7-9f8bc33a06ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:10.497169 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-1b0667be-2bb9-49de-ba8e-f11080f05e86 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:10.499374 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-1b0667be-2bb9-49de-ba8e-f11080f05e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:10.505772 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe49d96d-0445-4019-95d7-9f8bc33a06ba admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:10.508558 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1323/2658] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:03:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:10.511791 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a2528-933e-4377-b3dd-5db864ac00bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:10.512793 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a2528-933e-4377-b3dd-5db864ac00bd admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:10.513233 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a2528-933e-4377-b3dd-5db864ac00bd admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:10.513676 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081a2528-933e-4377-b3dd-5db864ac00bd admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:10.514492 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a2528-933e-4377-b3dd-5db864ac00bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:10.516630 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeae202-ede6-4a18-b91d-49f7ed2e0660 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:10.516924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeae202-ede6-4a18-b91d-49f7ed2e0660 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:10.517078 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081a2528-933e-4377-b3dd-5db864ac00bd admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:10.517200 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeae202-ede6-4a18-b91d-49f7ed2e0660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:10.540798 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-9aeae202-ede6-4a18-b91d-49f7ed2e0660 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:10.542202 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-9aeae202-ede6-4a18-b91d-49f7ed2e0660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:10.552499 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1324/2659] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:03:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:10.557219 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeae202-ede6-4a18-b91d-49f7ed2e0660 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:10.567915 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeae202-ede6-4a18-b91d-49f7ed2e0660 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:10.574013 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b73cc-e0d8-4767-ab6b-db489920ca98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:10.574758 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b73cc-e0d8-4767-ab6b-db489920ca98 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:10.575615 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b73cc-e0d8-4767-ab6b-db489920ca98 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:10.575615 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0b73cc-e0d8-4767-ab6b-db489920ca98 admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:10.576617 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b73cc-e0d8-4767-ab6b-db489920ca98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:03:10.577441 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b73cc-e0d8-4767-ab6b-db489920ca98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:10.579103 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0b73cc-e0d8-4767-ab6b-db489920ca98 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:10.596470 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1325/2660] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 11:03:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:03:10.598693 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b73cc-e0d8-4767-ab6b-db489920ca98 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:10.609980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0b73cc-e0d8-4767-ab6b-db489920ca98 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:10.620811 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0e847-fe33-4a54-b65c-da63eebf32d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:10.621757 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0e847-fe33-4a54-b65c-da63eebf32d8 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:10.622186 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0e847-fe33-4a54-b65c-da63eebf32d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:10.622559 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf0e847-fe33-4a54-b65c-da63eebf32d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:10.623354 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf0e847-fe33-4a54-b65c-da63eebf32d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:10.625539 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf0e847-fe33-4a54-b65c-da63eebf32d8 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:10.681419 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1336/2661] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:03:10 2022] POST /identity/v3/projects => generated 372 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:03:10.688745 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a2528-933e-4377-b3dd-5db864ac00bd admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:10.734563 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1326/2662] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:03:10 2022] POST /identity/v3/projects => generated 384 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:03:10.745197 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0e847-fe33-4a54-b65c-da63eebf32d8 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:10.811206 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081a2528-933e-4377-b3dd-5db864ac00bd admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:10.856930 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf0e847-fe33-4a54-b65c-da63eebf32d8 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:10.861885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba21ce-b672-4c66-bab4-bd58c602f3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:10.863232 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba21ce-b672-4c66-bab4-bd58c602f3b3 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:10.866395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba21ce-b672-4c66-bab4-bd58c602f3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:10.866395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba21ce-b672-4c66-bab4-bd58c602f3b3 admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:10.866395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba21ce-b672-4c66-bab4-bd58c602f3b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:10.866736 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba21ce-b672-4c66-bab4-bd58c602f3b3 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:10.874945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768370f-4d4b-4937-ae3e-fa650e984547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:10.875606 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768370f-4d4b-4937-ae3e-fa650e984547 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:10.876148 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768370f-4d4b-4937-ae3e-fa650e984547 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:10.876148 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d768370f-4d4b-4937-ae3e-fa650e984547 admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:10.876806 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768370f-4d4b-4937-ae3e-fa650e984547 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:10.879552 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d768370f-4d4b-4937-ae3e-fa650e984547 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:10.961763 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1327/2663] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:03:10 2022] POST /identity/v3/users => generated 293 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:03:10.965334 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba21ce-b672-4c66-bab4-bd58c602f3b3 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:10.976516 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba21ce-b672-4c66-bab4-bd58c602f3b3 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:10.982188 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbf7c7-1110-4cf6-91fc-701be723aa2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:10.983112 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbf7c7-1110-4cf6-91fc-701be723aa2c admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:10.983551 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbf7c7-1110-4cf6-91fc-701be723aa2c admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:10.983678 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:10.984136 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cbf7c7-1110-4cf6-91fc-701be723aa2c admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:10.984862 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cbf7c7-1110-4cf6-91fc-701be723aa2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:10.986459 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cbf7c7-1110-4cf6-91fc-701be723aa2c admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:10.990399 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1337/2664] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:03:10 2022] POST /identity/v3/users => generated 287 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:03:10.996204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768370f-4d4b-4937-ae3e-fa650e984547 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:11.000013 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1328/2665] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:03:10 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:03:11.008051 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbf7c7-1110-4cf6-91fc-701be723aa2c admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:11.011297 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d768370f-4d4b-4937-ae3e-fa650e984547 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:11.017015 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746db6ac-60c1-4d4a-8495-9b602d26e79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:11.017651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746db6ac-60c1-4d4a-8495-9b602d26e79b admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:11.017932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746db6ac-60c1-4d4a-8495-9b602d26e79b admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:11.018376 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746db6ac-60c1-4d4a-8495-9b602d26e79b admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:11.019589 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746db6ac-60c1-4d4a-8495-9b602d26e79b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:11.021335 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746db6ac-60c1-4d4a-8495-9b602d26e79b admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:11.021454 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cbf7c7-1110-4cf6-91fc-701be723aa2c admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:11.027606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ab924-0c1f-4398-b4fb-d40ed9115015 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:11.028607 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ab924-0c1f-4398-b4fb-d40ed9115015 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:11.028959 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ab924-0c1f-4398-b4fb-d40ed9115015 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:11.030352 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ab924-0c1f-4398-b4fb-d40ed9115015 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:11.031123 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ab924-0c1f-4398-b4fb-d40ed9115015 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:11.032547 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1338/2666] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:03:10 2022] GET /identity/v3/roles => generated 1427 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:03:11.033299 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ab924-0c1f-4398-b4fb-d40ed9115015 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:11.037029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746db6ac-60c1-4d4a-8495-9b602d26e79b admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:11.047576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1329/2667] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:03:11 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:03:11.050783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746db6ac-60c1-4d4a-8495-9b602d26e79b admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:11.052674 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a54b5a-1eef-40a5-8dfa-a3b5465c2b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:11.053290 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a54b5a-1eef-40a5-8dfa-a3b5465c2b6d admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:11.053771 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a54b5a-1eef-40a5-8dfa-a3b5465c2b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:11.053986 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ab924-0c1f-4398-b4fb-d40ed9115015 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:11.054109 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a54b5a-1eef-40a5-8dfa-a3b5465c2b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:11.054804 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a54b5a-1eef-40a5-8dfa-a3b5465c2b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:11.056363 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a54b5a-1eef-40a5-8dfa-a3b5465c2b6d admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:11.064361 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ab924-0c1f-4398-b4fb-d40ed9115015 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:11.068826 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1339/2668] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:03:11 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:03:11.074703 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a54b5a-1eef-40a5-8dfa-a3b5465c2b6d admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:11.075262 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41579eb9-3a52-420b-af50-d9bac0130abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:11.076295 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41579eb9-3a52-420b-af50-d9bac0130abd admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:11.076746 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41579eb9-3a52-420b-af50-d9bac0130abd admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:11.077179 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41579eb9-3a52-420b-af50-d9bac0130abd admin admin] PATH_INFO: `/v3/projects/6900bd4be3a149c39ba57ef1578c9394/users/2266db60abd149ca95de4ab4c1aa2d12/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:11.091236 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a54b5a-1eef-40a5-8dfa-a3b5465c2b6d admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:11.098004 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41579eb9-3a52-420b-af50-d9bac0130abd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6900bd4be3a149c39ba57ef1578c9394, user_id=2266db60abd149ca95de4ab4c1aa2d12, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:11.100770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41579eb9-3a52-420b-af50-d9bac0130abd admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:11.114228 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d846a-b3dd-469a-8c90-1a82c37bc440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:11.114228 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d846a-b3dd-469a-8c90-1a82c37bc440 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:11.114228 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d846a-b3dd-469a-8c90-1a82c37bc440 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:11.114228 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d846a-b3dd-469a-8c90-1a82c37bc440 admin admin] PATH_INFO: `/v3/projects/9aa31b0e917d4c3990ead47bc92a2254/users/65618737312b47cd8c1ba3b822aff6ab/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:11.136847 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d846a-b3dd-469a-8c90-1a82c37bc440 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9aa31b0e917d4c3990ead47bc92a2254, user_id=65618737312b47cd8c1ba3b822aff6ab, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:11.141072 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531d846a-b3dd-469a-8c90-1a82c37bc440 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:11.177512 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1330/2669] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:03:11 2022] PUT /identity/v3/projects/6900bd4be3a149c39ba57ef1578c9394/users/2266db60abd149ca95de4ab4c1aa2d12/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:11.252218 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f38eb-555f-453a-a372-f8880cbfac32 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:11.252539 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f38eb-555f-453a-a372-f8880cbfac32 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:11.252842 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308f38eb-555f-453a-a372-f8880cbfac32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:11.276511 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1340/2670] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:03:11 2022] PUT /identity/v3/projects/9aa31b0e917d4c3990ead47bc92a2254/users/65618737312b47cd8c1ba3b822aff6ab/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:03:11.283433 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-308f38eb-555f-453a-a372-f8880cbfac32 None None] MFA Rules not processed for user `2266db60abd149ca95de4ab4c1aa2d12`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:11.285398 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-308f38eb-555f-453a-a372-f8880cbfac32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:11.353539 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92389a9a-474e-4972-80d7-e29781ff29ca None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:11.353833 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92389a9a-474e-4972-80d7-e29781ff29ca None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:11.354115 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92389a9a-474e-4972-80d7-e29781ff29ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:11.370976 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1331/2671] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:03:11 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:11.376461 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:11.384414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f38eb-555f-453a-a372-f8880cbfac32 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:11.388023 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-308f38eb-555f-453a-a372-f8880cbfac32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:11.393414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-92389a9a-474e-4972-80d7-e29781ff29ca None None] MFA Rules not processed for user `65618737312b47cd8c1ba3b822aff6ab`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:11.395517 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-92389a9a-474e-4972-80d7-e29781ff29ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:11.458647 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308f38eb-555f-453a-a372-f8880cbfac32 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:11.462656 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc86ab0-cbd7-483e-b8eb-a151c2da44d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:11.463341 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc86ab0-cbd7-483e-b8eb-a151c2da44d4 service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:11.463627 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc86ab0-cbd7-483e-b8eb-a151c2da44d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:11.463905 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc86ab0-cbd7-483e-b8eb-a151c2da44d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:11.474789 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc86ab0-cbd7-483e-b8eb-a151c2da44d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:11.476397 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc86ab0-cbd7-483e-b8eb-a151c2da44d4 service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:11.478898 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1341/2672] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:03:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:03:11.498788 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1332/2673] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:03:11 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:03:11.499583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92389a9a-474e-4972-80d7-e29781ff29ca None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:11.499583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-92389a9a-474e-4972-80d7-e29781ff29ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:11.564344 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92389a9a-474e-4972-80d7-e29781ff29ca None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:11.570189 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfcc9f9-0e41-4e97-8b6a-3882ddb37ce1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e358fd7c1b3c43d7ac89fb3562678266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:11.570752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcc9f9-0e41-4e97-8b6a-3882ddb37ce1 service glance] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:11.571032 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcc9f9-0e41-4e97-8b6a-3882ddb37ce1 service glance] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:11.571334 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfcc9f9-0e41-4e97-8b6a-3882ddb37ce1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:11.581990 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfcc9f9-0e41-4e97-8b6a-3882ddb37ce1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:11.583487 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfcc9f9-0e41-4e97-8b6a-3882ddb37ce1 service glance] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:11.596354 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1342/2674] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:03:11 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:03:13.184079 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:13.360899 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=91,keystone:INSERT=6,keystone:DELETE=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:16.000154 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc86ab0-cbd7-483e-b8eb-a151c2da44d4 service cinder] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:16.118694 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc86ab0-cbd7-483e-b8eb-a151c2da44d4 service cinder] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:16.124838 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b68ddc-fd2a-42af-b7c1-2200f8a47672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:16.125530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b68ddc-fd2a-42af-b7c1-2200f8a47672 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:16.125817 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b68ddc-fd2a-42af-b7c1-2200f8a47672 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:16.126093 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b68ddc-fd2a-42af-b7c1-2200f8a47672 admin admin] PATH_INFO: `/v3/users/2266db60abd149ca95de4ab4c1aa2d12` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:16.127399 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b68ddc-fd2a-42af-b7c1-2200f8a47672 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2266db60abd149ca95de4ab4c1aa2d12)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:16.131327 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b68ddc-fd2a-42af-b7c1-2200f8a47672 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:16.376504 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-f2b68ddc-fd2a-42af-b7c1-2200f8a47672 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2266db60abd149ca95de4ab4c1aa2d12'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:16.377461 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-f2b68ddc-fd2a-42af-b7c1-2200f8a47672 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2266db60abd149ca95de4ab4c1aa2d12'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:16.469857 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-f2b68ddc-fd2a-42af-b7c1-2200f8a47672 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2266db60abd149ca95de4ab4c1aa2d12'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:16.479829 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-f2b68ddc-fd2a-42af-b7c1-2200f8a47672 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2266db60abd149ca95de4ab4c1aa2d12'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:16.487691 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:16.498472 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-f2b68ddc-fd2a-42af-b7c1-2200f8a47672 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2266db60abd149ca95de4ab4c1aa2d12'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:16.508539 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1333/2675] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:03:15 2022] DELETE /identity/v3/users/2266db60abd149ca95de4ab4c1aa2d12 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:16.513196 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b8b80-c3e5-4bdf-a69d-f9735d12e068 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:16.513682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b8b80-c3e5-4bdf-a69d-f9735d12e068 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:16.513790 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b8b80-c3e5-4bdf-a69d-f9735d12e068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:16.535469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-c35b8b80-c3e5-4bdf-a69d-f9735d12e068 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:16.536703 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-c35b8b80-c3e5-4bdf-a69d-f9735d12e068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:16.571114 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:16.605558 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1343/2676] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:03:16 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:03:16.616839 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b68ddc-fd2a-42af-b7c1-2200f8a47672 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:16.617880 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-f2b68ddc-fd2a-42af-b7c1-2200f8a47672 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:16.679086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b68ddc-fd2a-42af-b7c1-2200f8a47672 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:16.682883 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee29ed-409b-4873-ad49-a75d9cdeb0db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:16.683531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee29ed-409b-4873-ad49-a75d9cdeb0db service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:16.683818 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee29ed-409b-4873-ad49-a75d9cdeb0db service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:16.684097 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee29ed-409b-4873-ad49-a75d9cdeb0db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:16.695907 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee29ed-409b-4873-ad49-a75d9cdeb0db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:16.697666 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee29ed-409b-4873-ad49-a75d9cdeb0db service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:16.709638 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1334/2677] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:03:16 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:03:17.173819 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b8b80-c3e5-4bdf-a69d-f9735d12e068 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:17.176176 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-c35b8b80-c3e5-4bdf-a69d-f9735d12e068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:17.203221 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b8b80-c3e5-4bdf-a69d-f9735d12e068 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:17.210168 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71c7da-b177-4bce-bcc1-e582076c322d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:17.214415 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71c7da-b177-4bce-bcc1-e582076c322d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:17.215298 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71c7da-b177-4bce-bcc1-e582076c322d admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:17.218354 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71c7da-b177-4bce-bcc1-e582076c322d admin admin] PATH_INFO: `/v3/users/2664b70cbc1a41a59f78b0b7d4e86643` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:17.218354 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e71c7da-b177-4bce-bcc1-e582076c322d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2664b70cbc1a41a59f78b0b7d4e86643)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:17.221827 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e71c7da-b177-4bce-bcc1-e582076c322d admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:17.428443 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee29ed-409b-4873-ad49-a75d9cdeb0db service neutron] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:17.429964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-d4ee29ed-409b-4873-ad49-a75d9cdeb0db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:17.445282 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee29ed-409b-4873-ad49-a75d9cdeb0db service neutron] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:17.450814 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5f0ae5-a824-4ffa-906c-fc992039f564 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:17.452224 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f0ae5-a824-4ffa-906c-fc992039f564 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:17.456341 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f0ae5-a824-4ffa-906c-fc992039f564 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:17.456341 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5f0ae5-a824-4ffa-906c-fc992039f564 admin admin] PATH_INFO: `/v3/projects/6900bd4be3a149c39ba57ef1578c9394` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:17.456341 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f0ae5-a824-4ffa-906c-fc992039f564 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6900bd4be3a149c39ba57ef1578c9394)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:17.457279 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5f0ae5-a824-4ffa-906c-fc992039f564 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:17.495896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-7e71c7da-b177-4bce-bcc1-e582076c322d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2664b70cbc1a41a59f78b0b7d4e86643'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:17.504346 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-7e71c7da-b177-4bce-bcc1-e582076c322d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2664b70cbc1a41a59f78b0b7d4e86643'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:17.505235 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-7e71c7da-b177-4bce-bcc1-e582076c322d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2664b70cbc1a41a59f78b0b7d4e86643'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:17.635695 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-cc5f0ae5-a824-4ffa-906c-fc992039f564 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6900bd4be3a149c39ba57ef1578c9394', 'request_id': 'req-cc5f0ae5-a824-4ffa-906c-fc992039f564'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:17.641851 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-cc5f0ae5-a824-4ffa-906c-fc992039f564 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6900bd4be3a149c39ba57ef1578c9394', 'request_id': 'req-cc5f0ae5-a824-4ffa-906c-fc992039f564'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:17.653482 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-7e71c7da-b177-4bce-bcc1-e582076c322d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2664b70cbc1a41a59f78b0b7d4e86643'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:17.669745 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-7e71c7da-b177-4bce-bcc1-e582076c322d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2664b70cbc1a41a59f78b0b7d4e86643'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:17.677149 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1344/2678] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:03:17 2022] DELETE /identity/v3/users/2664b70cbc1a41a59f78b0b7d4e86643 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:03:17.795987 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-cc5f0ae5-a824-4ffa-906c-fc992039f564 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6900bd4be3a149c39ba57ef1578c9394', 'request_id': 'req-cc5f0ae5-a824-4ffa-906c-fc992039f564'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:17.808028 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-cc5f0ae5-a824-4ffa-906c-fc992039f564 admin admin] The token cache is being invalidate because project 6900bd4be3a149c39ba57ef1578c9394 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:03:17.808704 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-cc5f0ae5-a824-4ffa-906c-fc992039f564 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:17.809839 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-cc5f0ae5-a824-4ffa-906c-fc992039f564 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:17.812274 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1335/2679] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:03:17 2022] DELETE /identity/v3/projects/6900bd4be3a149c39ba57ef1578c9394 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:03:17.818707 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f140d4-1845-43c6-bd77-a9236291da08 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:17.818707 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f140d4-1845-43c6-bd77-a9236291da08 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:17.818707 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f140d4-1845-43c6-bd77-a9236291da08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:17.842828 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-29f140d4-1845-43c6-bd77-a9236291da08 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:17.844349 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-29f140d4-1845-43c6-bd77-a9236291da08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:17.911301 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1345/2680] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:03:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:03:17.919086 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e9a19-b689-4973-be2d-44bdf969e0a1 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:17.919385 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e9a19-b689-4973-be2d-44bdf969e0a1 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:17.919674 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e9a19-b689-4973-be2d-44bdf969e0a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:17.941156 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-730e9a19-b689-4973-be2d-44bdf969e0a1 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:17.942380 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-730e9a19-b689-4973-be2d-44bdf969e0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:17.954068 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1336/2681] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:03:17 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:17.961958 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f140d4-1845-43c6-bd77-a9236291da08 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:17.972553 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f140d4-1845-43c6-bd77-a9236291da08 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:17.977155 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc40ed-f753-4bf8-a156-8dc18a37fea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:17.977756 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc40ed-f753-4bf8-a156-8dc18a37fea8 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:17.978028 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc40ed-f753-4bf8-a156-8dc18a37fea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:17.978464 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc40ed-f753-4bf8-a156-8dc18a37fea8 admin admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:17.979185 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc40ed-f753-4bf8-a156-8dc18a37fea8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:03:17.979482 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc40ed-f753-4bf8-a156-8dc18a37fea8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:17.981061 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc40ed-f753-4bf8-a156-8dc18a37fea8 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:17.999871 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e9a19-b689-4973-be2d-44bdf969e0a1 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:18.001153 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1346/2682] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 11:03:17 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:03:18.004862 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc40ed-f753-4bf8-a156-8dc18a37fea8 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:18.020161 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-730e9a19-b689-4973-be2d-44bdf969e0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:18.026580 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc40ed-f753-4bf8-a156-8dc18a37fea8 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:18.033612 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77719385-17ee-4b0d-9ef6-9d33bf116e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:18.033612 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77719385-17ee-4b0d-9ef6-9d33bf116e5d admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:18.033612 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77719385-17ee-4b0d-9ef6-9d33bf116e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:18.033612 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77719385-17ee-4b0d-9ef6-9d33bf116e5d admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:18.034759 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77719385-17ee-4b0d-9ef6-9d33bf116e5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:18.037710 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77719385-17ee-4b0d-9ef6-9d33bf116e5d admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:18.044700 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e9a19-b689-4973-be2d-44bdf969e0a1 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:18.051951 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d7d8e-9ca2-4452-a276-d016f1b54162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:18.052966 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d7d8e-9ca2-4452-a276-d016f1b54162 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:18.053489 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d7d8e-9ca2-4452-a276-d016f1b54162 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:18.054143 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751d7d8e-9ca2-4452-a276-d016f1b54162 admin admin] PATH_INFO: `/v3/projects/cd54719e5f2b41e6a875361a9188d49b` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:18.055670 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d7d8e-9ca2-4452-a276-d016f1b54162 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd54719e5f2b41e6a875361a9188d49b)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:18.058672 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751d7d8e-9ca2-4452-a276-d016f1b54162 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:18.146445 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1347/2683] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:03:18 2022] POST /identity/v3/projects => generated 382 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:03:18.151376 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77719385-17ee-4b0d-9ef6-9d33bf116e5d admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:18.165714 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:18.219118 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77719385-17ee-4b0d-9ef6-9d33bf116e5d admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:18.233649 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-751d7d8e-9ca2-4452-a276-d016f1b54162 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd54719e5f2b41e6a875361a9188d49b', 'request_id': 'req-751d7d8e-9ca2-4452-a276-d016f1b54162'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:18.241268 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-751d7d8e-9ca2-4452-a276-d016f1b54162 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd54719e5f2b41e6a875361a9188d49b', 'request_id': 'req-751d7d8e-9ca2-4452-a276-d016f1b54162'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:18.280394 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f531c-2f19-4d06-ad6b-04b0ff6c7262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:18.281057 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f531c-2f19-4d06-ad6b-04b0ff6c7262 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:18.281338 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f531c-2f19-4d06-ad6b-04b0ff6c7262 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:18.281702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f531c-2f19-4d06-ad6b-04b0ff6c7262 admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:18.282380 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f531c-2f19-4d06-ad6b-04b0ff6c7262 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:18.285070 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643f531c-2f19-4d06-ad6b-04b0ff6c7262 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:18.315126 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-751d7d8e-9ca2-4452-a276-d016f1b54162 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd54719e5f2b41e6a875361a9188d49b', 'request_id': 'req-751d7d8e-9ca2-4452-a276-d016f1b54162'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:18.330287 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-751d7d8e-9ca2-4452-a276-d016f1b54162 admin admin] The token cache is being invalidate because project cd54719e5f2b41e6a875361a9188d49b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:03:18.331146 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-751d7d8e-9ca2-4452-a276-d016f1b54162 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:18.331801 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-751d7d8e-9ca2-4452-a276-d016f1b54162 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:18.333659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1337/2684] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:03:17 2022] DELETE /identity/v3/projects/cd54719e5f2b41e6a875361a9188d49b => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:03:18.337333 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d7d8e-9ca2-4452-a276-d016f1b54162 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:18.338706 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-751d7d8e-9ca2-4452-a276-d016f1b54162 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:18.354205 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751d7d8e-9ca2-4452-a276-d016f1b54162 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:18.358960 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80caf27d-475a-44e8-9010-59dd4f67bccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:18.359650 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80caf27d-475a-44e8-9010-59dd4f67bccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:18.359959 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80caf27d-475a-44e8-9010-59dd4f67bccd admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:18.360239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80caf27d-475a-44e8-9010-59dd4f67bccd admin admin] PATH_INFO: `/v3/users/8452abedb4a74e4cad5e804ccd9b8a03` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:18.361410 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80caf27d-475a-44e8-9010-59dd4f67bccd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8452abedb4a74e4cad5e804ccd9b8a03)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:18.363624 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80caf27d-475a-44e8-9010-59dd4f67bccd admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:18.424457 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1348/2685] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:03:18 2022] POST /identity/v3/users => generated 292 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:03:18.428657 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f531c-2f19-4d06-ad6b-04b0ff6c7262 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:18.430460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-643f531c-2f19-4d06-ad6b-04b0ff6c7262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:18.444460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643f531c-2f19-4d06-ad6b-04b0ff6c7262 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:18.451516 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964d62e-7556-411f-81aa-dfe5c58420cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:18.451516 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964d62e-7556-411f-81aa-dfe5c58420cb admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:18.451516 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964d62e-7556-411f-81aa-dfe5c58420cb admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:18.458154 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9964d62e-7556-411f-81aa-dfe5c58420cb admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:18.458154 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9964d62e-7556-411f-81aa-dfe5c58420cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:18.458154 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9964d62e-7556-411f-81aa-dfe5c58420cb admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:18.465809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1349/2686] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:03:18 2022] GET /identity/v3/roles => generated 1427 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:03:18.469645 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964d62e-7556-411f-81aa-dfe5c58420cb admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:18.479307 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9964d62e-7556-411f-81aa-dfe5c58420cb admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:18.484444 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859176e-cfd3-4fcf-b256-76e938ebbf1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:18.485071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859176e-cfd3-4fcf-b256-76e938ebbf1c admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:18.485362 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859176e-cfd3-4fcf-b256-76e938ebbf1c admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:18.485646 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0859176e-cfd3-4fcf-b256-76e938ebbf1c admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:18.486381 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0859176e-cfd3-4fcf-b256-76e938ebbf1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:18.488152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0859176e-cfd3-4fcf-b256-76e938ebbf1c admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:18.499472 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1350/2687] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:03:18 2022] GET /identity/v3/roles => generated 1427 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:03:18.504086 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859176e-cfd3-4fcf-b256-76e938ebbf1c admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:18.514680 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0859176e-cfd3-4fcf-b256-76e938ebbf1c admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:18.520798 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c6d51d-0cdd-4a24-a3d9-d5ac315e81c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:18.521760 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6d51d-0cdd-4a24-a3d9-d5ac315e81c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:18.521913 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:18.522347 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6d51d-0cdd-4a24-a3d9-d5ac315e81c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:18.522347 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c6d51d-0cdd-4a24-a3d9-d5ac315e81c8 admin admin] PATH_INFO: `/v3/projects/1ebdffccaaa04484b32602a67fe23629/users/6bf80e69ea524d83ab199fc7315f4be2/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:18.537127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c6d51d-0cdd-4a24-a3d9-d5ac315e81c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ebdffccaaa04484b32602a67fe23629, user_id=6bf80e69ea524d83ab199fc7315f4be2, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:18.539879 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c6d51d-0cdd-4a24-a3d9-d5ac315e81c8 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:18.633962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1351/2688] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:03:18 2022] PUT /identity/v3/projects/1ebdffccaaa04484b32602a67fe23629/users/6bf80e69ea524d83ab199fc7315f4be2/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:03:18.635468 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f43cc9-d4f1-408f-b2c7-837d974a25c7 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:18.635752 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f43cc9-d4f1-408f-b2c7-837d974a25c7 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:18.636031 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f43cc9-d4f1-408f-b2c7-837d974a25c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:18.641431 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-80caf27d-475a-44e8-9010-59dd4f67bccd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8452abedb4a74e4cad5e804ccd9b8a03'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:18.642273 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-80caf27d-475a-44e8-9010-59dd4f67bccd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8452abedb4a74e4cad5e804ccd9b8a03'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:18.656127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-e4f43cc9-d4f1-408f-b2c7-837d974a25c7 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:18.657361 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-e4f43cc9-d4f1-408f-b2c7-837d974a25c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:18.717476 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-80caf27d-475a-44e8-9010-59dd4f67bccd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8452abedb4a74e4cad5e804ccd9b8a03'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:18.719249 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1352/2689] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:03:18 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:18.725731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-80caf27d-475a-44e8-9010-59dd4f67bccd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8452abedb4a74e4cad5e804ccd9b8a03'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:18.729791 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f43cc9-d4f1-408f-b2c7-837d974a25c7 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:18.731427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-e4f43cc9-d4f1-408f-b2c7-837d974a25c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:18.744300 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-80caf27d-475a-44e8-9010-59dd4f67bccd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8452abedb4a74e4cad5e804ccd9b8a03'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:18.756434 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1338/2690] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:03:18 2022] DELETE /identity/v3/users/8452abedb4a74e4cad5e804ccd9b8a03 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:03:18.810512 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f43cc9-d4f1-408f-b2c7-837d974a25c7 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:18.870019 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d9cbf-eb9d-4ae2-bf2d-85a83eaa0d1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:18.870799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d9cbf-eb9d-4ae2-bf2d-85a83eaa0d1b service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:18.871152 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d9cbf-eb9d-4ae2-bf2d-85a83eaa0d1b service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:18.871504 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d9cbf-eb9d-4ae2-bf2d-85a83eaa0d1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:18.873305 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-db5d9cbf-eb9d-4ae2-bf2d-85a83eaa0d1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:18.956577 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5d9cbf-eb9d-4ae2-bf2d-85a83eaa0d1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:18.958092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5d9cbf-eb9d-4ae2-bf2d-85a83eaa0d1b service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:18.968614 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1353/2691] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:03:18 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:03:19.031424 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80caf27d-475a-44e8-9010-59dd4f67bccd admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:19.032901 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-80caf27d-475a-44e8-9010-59dd4f67bccd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:19.046853 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80caf27d-475a-44e8-9010-59dd4f67bccd admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:19.052906 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd02d3c-966c-448c-bf31-0d0b4b3288e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:19.053666 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd02d3c-966c-448c-bf31-0d0b4b3288e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:19.053950 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd02d3c-966c-448c-bf31-0d0b4b3288e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:19.054227 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd02d3c-966c-448c-bf31-0d0b4b3288e5 admin admin] PATH_INFO: `/v3/projects/4cf6a5aa45e14f8d96889a057e0c6cfe` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:19.055566 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd02d3c-966c-448c-bf31-0d0b4b3288e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cf6a5aa45e14f8d96889a057e0c6cfe)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:19.063171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd02d3c-966c-448c-bf31-0d0b4b3288e5 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:19.243718 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-8fd02d3c-966c-448c-bf31-0d0b4b3288e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cf6a5aa45e14f8d96889a057e0c6cfe', 'request_id': 'req-8fd02d3c-966c-448c-bf31-0d0b4b3288e5'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:19.272977 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-8fd02d3c-966c-448c-bf31-0d0b4b3288e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cf6a5aa45e14f8d96889a057e0c6cfe', 'request_id': 'req-8fd02d3c-966c-448c-bf31-0d0b4b3288e5'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:19.356443 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-8fd02d3c-966c-448c-bf31-0d0b4b3288e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cf6a5aa45e14f8d96889a057e0c6cfe', 'request_id': 'req-8fd02d3c-966c-448c-bf31-0d0b4b3288e5'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:19.375738 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-8fd02d3c-966c-448c-bf31-0d0b4b3288e5 admin admin] The token cache is being invalidate because project 4cf6a5aa45e14f8d96889a057e0c6cfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:03:19.375894 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-8fd02d3c-966c-448c-bf31-0d0b4b3288e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:19.376791 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-8fd02d3c-966c-448c-bf31-0d0b4b3288e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:19.384460 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1339/2692] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:03:19 2022] DELETE /identity/v3/projects/4cf6a5aa45e14f8d96889a057e0c6cfe => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:03:19.549251 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d9cbf-eb9d-4ae2-bf2d-85a83eaa0d1b service neutron] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:19.551506 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-db5d9cbf-eb9d-4ae2-bf2d-85a83eaa0d1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:19.565397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d9cbf-eb9d-4ae2-bf2d-85a83eaa0d1b service neutron] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:19.649201 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66972d0f-1fa0-44f8-ab24-8f64ed089229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:19.649903 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66972d0f-1fa0-44f8-ab24-8f64ed089229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:19.650293 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66972d0f-1fa0-44f8-ab24-8f64ed089229 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:19.650611 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66972d0f-1fa0-44f8-ab24-8f64ed089229 admin admin] PATH_INFO: `/v3/users/ab4b8cf9f04940428a8792a09f824378` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:19.651850 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66972d0f-1fa0-44f8-ab24-8f64ed089229 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab4b8cf9f04940428a8792a09f824378)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:19.653796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66972d0f-1fa0-44f8-ab24-8f64ed089229 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:19.852293 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:19.920417 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-66972d0f-1fa0-44f8-ab24-8f64ed089229 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab4b8cf9f04940428a8792a09f824378'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:19.931994 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-66972d0f-1fa0-44f8-ab24-8f64ed089229 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab4b8cf9f04940428a8792a09f824378'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:19.933704 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-66972d0f-1fa0-44f8-ab24-8f64ed089229 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab4b8cf9f04940428a8792a09f824378'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:20.046262 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-66972d0f-1fa0-44f8-ab24-8f64ed089229 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab4b8cf9f04940428a8792a09f824378'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:20.060122 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-66972d0f-1fa0-44f8-ab24-8f64ed089229 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab4b8cf9f04940428a8792a09f824378'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:20.067942 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1354/2693] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:03:19 2022] DELETE /identity/v3/users/ab4b8cf9f04940428a8792a09f824378 => generated 0 bytes in 519 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:20.072591 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eed76b-9336-47ad-aa3a-4b954a6411b0 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:20.073097 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eed76b-9336-47ad-aa3a-4b954a6411b0 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:20.073359 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eed76b-9336-47ad-aa3a-4b954a6411b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:20.093102 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-d7eed76b-9336-47ad-aa3a-4b954a6411b0 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:20.094325 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-d7eed76b-9336-47ad-aa3a-4b954a6411b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:20.139794 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:20.162483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1340/2694] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:03:20 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:20.178322 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66972d0f-1fa0-44f8-ab24-8f64ed089229 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:20.180550 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-66972d0f-1fa0-44f8-ab24-8f64ed089229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:20.259757 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66972d0f-1fa0-44f8-ab24-8f64ed089229 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:20.263437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94082394-1fe5-43d7-9df8-fb7a225e83c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:20.265309 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94082394-1fe5-43d7-9df8-fb7a225e83c0 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:20.265548 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94082394-1fe5-43d7-9df8-fb7a225e83c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:20.265891 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94082394-1fe5-43d7-9df8-fb7a225e83c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:20.284142 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94082394-1fe5-43d7-9df8-fb7a225e83c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:20.286156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94082394-1fe5-43d7-9df8-fb7a225e83c0 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:20.302567 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1355/2695] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:03:20 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:03:20.586572 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e6fc2-52ff-49d2-8e63-7403eac842ac None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:20.586907 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e6fc2-52ff-49d2-8e63-7403eac842ac None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:20.587155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048e6fc2-52ff-49d2-8e63-7403eac842ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:20.624280 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-048e6fc2-52ff-49d2-8e63-7403eac842ac None None] MFA Rules not processed for user `6bf80e69ea524d83ab199fc7315f4be2`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:20.625646 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-048e6fc2-52ff-49d2-8e63-7403eac842ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:20.700962 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1341/2696] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:03:20 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:20.712622 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94082394-1fe5-43d7-9df8-fb7a225e83c0 service neutron] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:20.715441 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-94082394-1fe5-43d7-9df8-fb7a225e83c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:20.750104 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=100 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:20.798936 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94082394-1fe5-43d7-9df8-fb7a225e83c0 service neutron] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:20.803946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81828de5-9e38-4da3-938c-1da1344c4f46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:20.804671 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81828de5-9e38-4da3-938c-1da1344c4f46 service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:20.804944 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81828de5-9e38-4da3-938c-1da1344c4f46 service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:20.805212 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81828de5-9e38-4da3-938c-1da1344c4f46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:20.814113 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81828de5-9e38-4da3-938c-1da1344c4f46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:20.816168 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81828de5-9e38-4da3-938c-1da1344c4f46 service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:20.825697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1356/2697] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:03:20 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:03:20.854162 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e6fc2-52ff-49d2-8e63-7403eac842ac None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:20.856359 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-048e6fc2-52ff-49d2-8e63-7403eac842ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:20.883382 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048e6fc2-52ff-49d2-8e63-7403eac842ac None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:20.888239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c51cb-18e6-49ba-9aa5-d36a66bf70a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:20.888934 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c51cb-18e6-49ba-9aa5-d36a66bf70a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:20.889158 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c51cb-18e6-49ba-9aa5-d36a66bf70a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:20.889443 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249c51cb-18e6-49ba-9aa5-d36a66bf70a0 admin admin] PATH_INFO: `/v3/projects/74000cc7fc2845628610ca8b17ae395b` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:20.890677 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c51cb-18e6-49ba-9aa5-d36a66bf70a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=74000cc7fc2845628610ca8b17ae395b)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:20.892710 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249c51cb-18e6-49ba-9aa5-d36a66bf70a0 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:21.015695 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-249c51cb-18e6-49ba-9aa5-d36a66bf70a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74000cc7fc2845628610ca8b17ae395b', 'request_id': 'req-249c51cb-18e6-49ba-9aa5-d36a66bf70a0'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:21.023737 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-249c51cb-18e6-49ba-9aa5-d36a66bf70a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74000cc7fc2845628610ca8b17ae395b', 'request_id': 'req-249c51cb-18e6-49ba-9aa5-d36a66bf70a0'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:21.118632 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-249c51cb-18e6-49ba-9aa5-d36a66bf70a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74000cc7fc2845628610ca8b17ae395b', 'request_id': 'req-249c51cb-18e6-49ba-9aa5-d36a66bf70a0'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:21.130927 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-249c51cb-18e6-49ba-9aa5-d36a66bf70a0 admin admin] The token cache is being invalidate because project 74000cc7fc2845628610ca8b17ae395b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:03:21.131237 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-249c51cb-18e6-49ba-9aa5-d36a66bf70a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:21.131886 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-249c51cb-18e6-49ba-9aa5-d36a66bf70a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:21.133489 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1342/2698] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:03:20 2022] DELETE /identity/v3/projects/74000cc7fc2845628610ca8b17ae395b => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:21.138197 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81828de5-9e38-4da3-938c-1da1344c4f46 service cinder] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:21.150803 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81828de5-9e38-4da3-938c-1da1344c4f46 service cinder] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:21.208995 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3f018-249d-4c73-a053-782fd8ae5c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:21.209656 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3f018-249d-4c73-a053-782fd8ae5c9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:21.209932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3f018-249d-4c73-a053-782fd8ae5c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:21.210204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3f018-249d-4c73-a053-782fd8ae5c9d admin admin] PATH_INFO: `/v3/users/7c250ba3b9f245629428a220b4143c4f` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:21.211415 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d3f018-249d-4c73-a053-782fd8ae5c9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c250ba3b9f245629428a220b4143c4f)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:21.213381 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d3f018-249d-4c73-a053-782fd8ae5c9d admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:21.429234 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:21.500376 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-b1d3f018-249d-4c73-a053-782fd8ae5c9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c250ba3b9f245629428a220b4143c4f'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:21.507876 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-b1d3f018-249d-4c73-a053-782fd8ae5c9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c250ba3b9f245629428a220b4143c4f'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:21.508565 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-b1d3f018-249d-4c73-a053-782fd8ae5c9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c250ba3b9f245629428a220b4143c4f'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:21.629893 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-b1d3f018-249d-4c73-a053-782fd8ae5c9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c250ba3b9f245629428a220b4143c4f'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:21.642026 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-b1d3f018-249d-4c73-a053-782fd8ae5c9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c250ba3b9f245629428a220b4143c4f'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:21.649134 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1357/2699] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:03:21 2022] DELETE /identity/v3/users/7c250ba3b9f245629428a220b4143c4f => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:22.120528 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c51cb-18e6-49ba-9aa5-d36a66bf70a0 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:22.122030 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-249c51cb-18e6-49ba-9aa5-d36a66bf70a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:22.138860 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249c51cb-18e6-49ba-9aa5-d36a66bf70a0 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:22.198945 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:22.206762 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff02331-2724-4bc0-8030-3007ae925d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:22.207436 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff02331-2724-4bc0-8030-3007ae925d6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:22.207712 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff02331-2724-4bc0-8030-3007ae925d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:22.207984 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff02331-2724-4bc0-8030-3007ae925d6c admin admin] PATH_INFO: `/v3/projects/b36e4e4fe12e4442b1e9b9d85d2b24ee` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:22.209119 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff02331-2724-4bc0-8030-3007ae925d6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b36e4e4fe12e4442b1e9b9d85d2b24ee)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:22.211024 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff02331-2724-4bc0-8030-3007ae925d6c admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:22.371282 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-dff02331-2724-4bc0-8030-3007ae925d6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b36e4e4fe12e4442b1e9b9d85d2b24ee', 'request_id': 'req-dff02331-2724-4bc0-8030-3007ae925d6c'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:22.378232 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-dff02331-2724-4bc0-8030-3007ae925d6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b36e4e4fe12e4442b1e9b9d85d2b24ee', 'request_id': 'req-dff02331-2724-4bc0-8030-3007ae925d6c'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:22.497509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-dff02331-2724-4bc0-8030-3007ae925d6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b36e4e4fe12e4442b1e9b9d85d2b24ee', 'request_id': 'req-dff02331-2724-4bc0-8030-3007ae925d6c'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:22.509047 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-dff02331-2724-4bc0-8030-3007ae925d6c admin admin] The token cache is being invalidate because project b36e4e4fe12e4442b1e9b9d85d2b24ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:03:22.509340 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-dff02331-2724-4bc0-8030-3007ae925d6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:22.509999 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-dff02331-2724-4bc0-8030-3007ae925d6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:03:22.511628 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1343/2700] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:03:22 2022] DELETE /identity/v3/projects/b36e4e4fe12e4442b1e9b9d85d2b24ee => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:22.517489 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ba7bb-de10-4d5a-8a18-b2c374018e11 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:22.519965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ba7bb-de10-4d5a-8a18-b2c374018e11 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:22.519965 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ba7bb-de10-4d5a-8a18-b2c374018e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:22.539935 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-0a7ba7bb-de10-4d5a-8a18-b2c374018e11 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:22.541174 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-0a7ba7bb-de10-4d5a-8a18-b2c374018e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:22.605874 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1358/2701] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:03:22 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:22.617033 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd1043-42ce-42af-9324-6657711c1404 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:22.617033 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd1043-42ce-42af-9324-6657711c1404 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:22.617033 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fd1043-42ce-42af-9324-6657711c1404 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:22.629227 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7d666-4cb3-4449-93ba-29097aa34e96 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:22.629587 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7d666-4cb3-4449-93ba-29097aa34e96 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:22.629932 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7d666-4cb3-4449-93ba-29097aa34e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:22.645223 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-87fd1043-42ce-42af-9324-6657711c1404 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:22.646804 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-87fd1043-42ce-42af-9324-6657711c1404 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:22.651792 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-38b7d666-4cb3-4449-93ba-29097aa34e96 None None] MFA Rules not processed for user `d4ba1b5dac82438391a56b7cda25a363`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:22.653287 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-38b7d666-4cb3-4449-93ba-29097aa34e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:22.660733 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1344/2702] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:03:22 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:03:22.661869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd1043-42ce-42af-9324-6657711c1404 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:22.670944 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fd1043-42ce-42af-9324-6657711c1404 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:22.685303 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba028714-29b9-4055-a15f-28dd7cb18bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:22.686468 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba028714-29b9-4055-a15f-28dd7cb18bf8 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:22.686468 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba028714-29b9-4055-a15f-28dd7cb18bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:22.687274 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba028714-29b9-4055-a15f-28dd7cb18bf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:22.688195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba028714-29b9-4055-a15f-28dd7cb18bf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:03:22.688539 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba028714-29b9-4055-a15f-28dd7cb18bf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:22.690177 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba028714-29b9-4055-a15f-28dd7cb18bf8 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:22.702918 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1345/2703] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 11:03:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:03:22.707707 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b5172-b8cb-4d2c-a5e2-f7018f86485f None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:22.707707 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b5172-b8cb-4d2c-a5e2-f7018f86485f None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:22.707707 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591b5172-b8cb-4d2c-a5e2-f7018f86485f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:22.724710 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-591b5172-b8cb-4d2c-a5e2-f7018f86485f None None] MFA Rules not processed for user `6bf80e69ea524d83ab199fc7315f4be2`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:22.725949 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-591b5172-b8cb-4d2c-a5e2-f7018f86485f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:22.728711 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1359/2704] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:03:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:22.729522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b7d666-4cb3-4449-93ba-29097aa34e96 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:22.739960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b7d666-4cb3-4449-93ba-29097aa34e96 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:22.744765 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30399cf-16ad-44e0-b59f-d938e80792a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:22.745386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30399cf-16ad-44e0-b59f-d938e80792a1 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:22.745670 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30399cf-16ad-44e0-b59f-d938e80792a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:22.747105 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30399cf-16ad-44e0-b59f-d938e80792a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:22.747105 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30399cf-16ad-44e0-b59f-d938e80792a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:22.751092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30399cf-16ad-44e0-b59f-d938e80792a1 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:22.801362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1346/2705] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:03:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:22.860429 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1360/2706] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:03:22 2022] POST /identity/v3/projects => generated 376 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:03:22.865534 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b5172-b8cb-4d2c-a5e2-f7018f86485f None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:22.877074 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591b5172-b8cb-4d2c-a5e2-f7018f86485f None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:22.938178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b6b84-9718-4397-ad6c-6b5103332375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:22.938836 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b6b84-9718-4397-ad6c-6b5103332375 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:22.939231 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b6b84-9718-4397-ad6c-6b5103332375 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:22.939506 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666b6b84-9718-4397-ad6c-6b5103332375 admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:22.940425 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b6b84-9718-4397-ad6c-6b5103332375 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:22.942509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666b6b84-9718-4397-ad6c-6b5103332375 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:22.981103 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:23.044324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1347/2707] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:03:22 2022] POST /identity/v3/users => generated 289 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:03:23.050953 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30399cf-16ad-44e0-b59f-d938e80792a1 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:23.061808 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30399cf-16ad-44e0-b59f-d938e80792a1 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:23.067959 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19030da-918d-45dd-9407-07ad37d6a2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:23.068652 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19030da-918d-45dd-9407-07ad37d6a2c7 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:23.068929 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19030da-918d-45dd-9407-07ad37d6a2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:23.069195 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19030da-918d-45dd-9407-07ad37d6a2c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:23.069883 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19030da-918d-45dd-9407-07ad37d6a2c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:23.071587 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19030da-918d-45dd-9407-07ad37d6a2c7 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:23.082847 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1361/2708] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:03:23 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:03:23.086967 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b6b84-9718-4397-ad6c-6b5103332375 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:23.097985 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666b6b84-9718-4397-ad6c-6b5103332375 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:23.102729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17ee751-7ffc-4c30-bd48-7741920e7b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:23.103496 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ee751-7ffc-4c30-bd48-7741920e7b9e admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:23.103837 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ee751-7ffc-4c30-bd48-7741920e7b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:23.104171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ee751-7ffc-4c30-bd48-7741920e7b9e admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:23.104928 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ee751-7ffc-4c30-bd48-7741920e7b9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:23.106582 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17ee751-7ffc-4c30-bd48-7741920e7b9e admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:23.118814 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1348/2709] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:03:23 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:03:23.123733 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19030da-918d-45dd-9407-07ad37d6a2c7 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:23.135033 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19030da-918d-45dd-9407-07ad37d6a2c7 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:23.139715 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e1a34-ec43-4420-90ee-6e2b17239cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:23.140378 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e1a34-ec43-4420-90ee-6e2b17239cbf admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:23.140706 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e1a34-ec43-4420-90ee-6e2b17239cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:23.140981 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1e1a34-ec43-4420-90ee-6e2b17239cbf admin admin] PATH_INFO: `/v3/projects/4060ad4dc3cf4855b563dc84dba99556/users/552719fa851145b89bd2858aea9a9e03/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:23.156717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1e1a34-ec43-4420-90ee-6e2b17239cbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4060ad4dc3cf4855b563dc84dba99556, user_id=552719fa851145b89bd2858aea9a9e03, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:23.160039 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1e1a34-ec43-4420-90ee-6e2b17239cbf admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:23.248354 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1362/2710] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:03:23 2022] PUT /identity/v3/projects/4060ad4dc3cf4855b563dc84dba99556/users/552719fa851145b89bd2858aea9a9e03/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:03:23.260510 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601bc08-82cb-48f4-90f5-3b5d3e1e8fdd None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:23.261013 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601bc08-82cb-48f4-90f5-3b5d3e1e8fdd None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:23.261163 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c601bc08-82cb-48f4-90f5-3b5d3e1e8fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:23.268959 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:23.295038 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-c601bc08-82cb-48f4-90f5-3b5d3e1e8fdd None None] MFA Rules not processed for user `552719fa851145b89bd2858aea9a9e03`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:23.296366 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-c601bc08-82cb-48f4-90f5-3b5d3e1e8fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:23.375640 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1349/2711] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:03:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:23.380695 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e1a34-ec43-4420-90ee-6e2b17239cbf admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:23.475833 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1e1a34-ec43-4420-90ee-6e2b17239cbf admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:23.480779 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb05f5f0-bafe-4a74-847d-87b85eead5f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:23.481402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05f5f0-bafe-4a74-847d-87b85eead5f1 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:23.481764 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05f5f0-bafe-4a74-847d-87b85eead5f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:23.482044 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb05f5f0-bafe-4a74-847d-87b85eead5f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:23.482856 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb05f5f0-bafe-4a74-847d-87b85eead5f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:23.484808 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb05f5f0-bafe-4a74-847d-87b85eead5f1 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:23.660572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1363/2712] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:03:23 2022] POST /identity/v3/projects => generated 378 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:03:23.664710 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c601bc08-82cb-48f4-90f5-3b5d3e1e8fdd None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:23.675832 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c601bc08-82cb-48f4-90f5-3b5d3e1e8fdd None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:23.749145 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d384b-f270-4ae0-8f6c-85037c27440d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:23.749989 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d384b-f270-4ae0-8f6c-85037c27440d admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:23.750485 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d384b-f270-4ae0-8f6c-85037c27440d admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:23.751765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092d384b-f270-4ae0-8f6c-85037c27440d admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:23.753159 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d384b-f270-4ae0-8f6c-85037c27440d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:23.755421 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092d384b-f270-4ae0-8f6c-85037c27440d admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:23.876017 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1350/2713] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:03:23 2022] POST /identity/v3/users => generated 296 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:03:23.878205 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb05f5f0-bafe-4a74-847d-87b85eead5f1 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:23.916993 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb05f5f0-bafe-4a74-847d-87b85eead5f1 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:23.940741 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70503996-f34d-49b4-8221-772b65d0775f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:23.941501 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70503996-f34d-49b4-8221-772b65d0775f admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:23.941770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70503996-f34d-49b4-8221-772b65d0775f admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:23.942231 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70503996-f34d-49b4-8221-772b65d0775f admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:23.943040 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70503996-f34d-49b4-8221-772b65d0775f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:23.948038 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70503996-f34d-49b4-8221-772b65d0775f admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:23.963991 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1364/2714] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:03:23 2022] GET /identity/v3/roles => generated 1427 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:03:23.968173 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d384b-f270-4ae0-8f6c-85037c27440d admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:23.981570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092d384b-f270-4ae0-8f6c-85037c27440d admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:23.987743 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216789b0-40bd-4ace-be3f-dbe98c73f42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:23.988537 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216789b0-40bd-4ace-be3f-dbe98c73f42a admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:23.988993 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216789b0-40bd-4ace-be3f-dbe98c73f42a admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:23.989421 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216789b0-40bd-4ace-be3f-dbe98c73f42a admin admin] PATH_INFO: `/v3/projects/b06e07371fdd43f480c7c29753d600a8/users/e8d8024c428b4de4ae95219511a2be12/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:24.009209 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216789b0-40bd-4ace-be3f-dbe98c73f42a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b06e07371fdd43f480c7c29753d600a8, user_id=e8d8024c428b4de4ae95219511a2be12, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:24.012075 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216789b0-40bd-4ace-be3f-dbe98c73f42a admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:24.126527 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1351/2715] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:03:23 2022] PUT /identity/v3/projects/b06e07371fdd43f480c7c29753d600a8/users/e8d8024c428b4de4ae95219511a2be12/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:03:24.131963 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70503996-f34d-49b4-8221-772b65d0775f admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:24.142707 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70503996-f34d-49b4-8221-772b65d0775f admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:24.195347 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:24.202044 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf2e5ea-9085-4ec4-aa50-a7c9852aa966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:24.202705 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2e5ea-9085-4ec4-aa50-a7c9852aa966 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:24.203005 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2e5ea-9085-4ec4-aa50-a7c9852aa966 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:24.203336 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf2e5ea-9085-4ec4-aa50-a7c9852aa966 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:24.204798 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2e5ea-9085-4ec4-aa50-a7c9852aa966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:24.205939 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf2e5ea-9085-4ec4-aa50-a7c9852aa966 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:24.217497 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1365/2716] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:03:24 2022] GET /identity/v3/roles => generated 1427 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:03:24.222045 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216789b0-40bd-4ace-be3f-dbe98c73f42a admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:24.232653 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216789b0-40bd-4ace-be3f-dbe98c73f42a admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:24.237870 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c58971-b2dc-4ec0-acaf-8bf29fccd2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:24.238894 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c58971-b2dc-4ec0-acaf-8bf29fccd2e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:24.238894 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c58971-b2dc-4ec0-acaf-8bf29fccd2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:24.239292 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c58971-b2dc-4ec0-acaf-8bf29fccd2e8 admin admin] PATH_INFO: `/v3/projects/b06e07371fdd43f480c7c29753d600a8/users/e8d8024c428b4de4ae95219511a2be12/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:24.241116 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c58971-b2dc-4ec0-acaf-8bf29fccd2e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b06e07371fdd43f480c7c29753d600a8, user_id=e8d8024c428b4de4ae95219511a2be12, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:24.243849 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c58971-b2dc-4ec0-acaf-8bf29fccd2e8 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:24.259107 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1352/2717] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:03:24 2022] PUT /identity/v3/projects/b06e07371fdd43f480c7c29753d600a8/users/e8d8024c428b4de4ae95219511a2be12/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:03:24.267462 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba0730-0a13-466c-a560-7d5b28903f57 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:24.267901 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba0730-0a13-466c-a560-7d5b28903f57 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:24.268260 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba0730-0a13-466c-a560-7d5b28903f57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:24.300790 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-45ba0730-0a13-466c-a560-7d5b28903f57 None None] MFA Rules not processed for user `e8d8024c428b4de4ae95219511a2be12`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:03:24.302595 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-45ba0730-0a13-466c-a560-7d5b28903f57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:24.390102 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1366/2718] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:03:24 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:03:24.402013 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c58971-b2dc-4ec0-acaf-8bf29fccd2e8 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:24.405944 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-70c58971-b2dc-4ec0-acaf-8bf29fccd2e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:24.446852 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:24.486492 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c58971-b2dc-4ec0-acaf-8bf29fccd2e8 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:24.491214 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f866639-af0d-4727-8802-6f47a671f363 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:24.491844 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f866639-af0d-4727-8802-6f47a671f363 service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:24.492134 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f866639-af0d-4727-8802-6f47a671f363 service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:24.492418 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f866639-af0d-4727-8802-6f47a671f363 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:24.564526 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f866639-af0d-4727-8802-6f47a671f363 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:24.566626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f866639-af0d-4727-8802-6f47a671f363 service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:24.575938 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1353/2719] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:03:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:03:32.198482 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba0730-0a13-466c-a560-7d5b28903f57 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:32.208669 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba0730-0a13-466c-a560-7d5b28903f57 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:32.212290 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ba1c36-bffe-4688-b37e-38c592f6026a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:32.212904 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba1c36-bffe-4688-b37e-38c592f6026a service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:32.213183 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba1c36-bffe-4688-b37e-38c592f6026a service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:32.213454 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ba1c36-bffe-4688-b37e-38c592f6026a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:32.223415 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba1c36-bffe-4688-b37e-38c592f6026a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:32.224845 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ba1c36-bffe-4688-b37e-38c592f6026a service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:03:32.234159 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1367/2720] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:03:32 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:03:34.615898 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=34 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:42.230047 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=44 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:03:59.718344 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f866639-af0d-4727-8802-6f47a671f363 service cinder] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:03:59.720383 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-8f866639-af0d-4727-8802-6f47a671f363 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:03:59.794201 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f866639-af0d-4727-8802-6f47a671f363 service cinder] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:03:59.798649 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f6207-8d78-41a1-8b78-9f99392b0652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:03:59.799317 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f6207-8d78-41a1-8b78-9f99392b0652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:03:59.799602 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f6207-8d78-41a1-8b78-9f99392b0652 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:03:59.799877 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9f6207-8d78-41a1-8b78-9f99392b0652 admin admin] PATH_INFO: `/v3/users/65618737312b47cd8c1ba3b822aff6ab` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:03:59.801830 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9f6207-8d78-41a1-8b78-9f99392b0652 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65618737312b47cd8c1ba3b822aff6ab)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:03:59.803775 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9f6207-8d78-41a1-8b78-9f99392b0652 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:00.012274 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-3b9f6207-8d78-41a1-8b78-9f99392b0652 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65618737312b47cd8c1ba3b822aff6ab'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:00.012921 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-3b9f6207-8d78-41a1-8b78-9f99392b0652 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65618737312b47cd8c1ba3b822aff6ab'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:00.073530 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-3b9f6207-8d78-41a1-8b78-9f99392b0652 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65618737312b47cd8c1ba3b822aff6ab'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:00.082190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-3b9f6207-8d78-41a1-8b78-9f99392b0652 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65618737312b47cd8c1ba3b822aff6ab'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:00.094865 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-3b9f6207-8d78-41a1-8b78-9f99392b0652 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65618737312b47cd8c1ba3b822aff6ab'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:00.101983 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1354/2721] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:03:59 2022] DELETE /identity/v3/users/65618737312b47cd8c1ba3b822aff6ab => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:04:00.106716 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc809a-34e9-4867-9e61-e678a616c3b8 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:00.107222 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc809a-34e9-4867-9e61-e678a616c3b8 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:00.107528 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc809a-34e9-4867-9e61-e678a616c3b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:00.129046 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-5efc809a-34e9-4867-9e61-e678a616c3b8 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:00.130288 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-5efc809a-34e9-4867-9e61-e678a616c3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:00.196612 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1368/2722] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:04:00 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:04:00.203969 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f6207-8d78-41a1-8b78-9f99392b0652 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:00.205734 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-3b9f6207-8d78-41a1-8b78-9f99392b0652 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:00.268597 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9f6207-8d78-41a1-8b78-9f99392b0652 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:00.273457 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d89819-d520-4ea3-986f-a42d2acf0a55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:00.274079 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d89819-d520-4ea3-986f-a42d2acf0a55 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:00.274426 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d89819-d520-4ea3-986f-a42d2acf0a55 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:00.274734 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d89819-d520-4ea3-986f-a42d2acf0a55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:00.284527 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d89819-d520-4ea3-986f-a42d2acf0a55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:00.286921 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d89819-d520-4ea3-986f-a42d2acf0a55 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:00.297454 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1355/2723] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:04:00 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:04:00.720601 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc809a-34e9-4867-9e61-e678a616c3b8 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:00.722219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-5efc809a-34e9-4867-9e61-e678a616c3b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:00.735036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc809a-34e9-4867-9e61-e678a616c3b8 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:00.740562 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce7b521-f6b8-4115-84a0-d86734b5a0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:00.741219 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7b521-f6b8-4115-84a0-d86734b5a0b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:00.741500 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7b521-f6b8-4115-84a0-d86734b5a0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:00.741791 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce7b521-f6b8-4115-84a0-d86734b5a0b0 admin admin] PATH_INFO: `/v3/projects/9aa31b0e917d4c3990ead47bc92a2254` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:00.743158 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce7b521-f6b8-4115-84a0-d86734b5a0b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9aa31b0e917d4c3990ead47bc92a2254)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:00.745499 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce7b521-f6b8-4115-84a0-d86734b5a0b0 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:00.855847 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-9ce7b521-f6b8-4115-84a0-d86734b5a0b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9aa31b0e917d4c3990ead47bc92a2254', 'request_id': 'req-9ce7b521-f6b8-4115-84a0-d86734b5a0b0'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:00.947823 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-9ce7b521-f6b8-4115-84a0-d86734b5a0b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9aa31b0e917d4c3990ead47bc92a2254', 'request_id': 'req-9ce7b521-f6b8-4115-84a0-d86734b5a0b0'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:00.955611 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-9ce7b521-f6b8-4115-84a0-d86734b5a0b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9aa31b0e917d4c3990ead47bc92a2254', 'request_id': 'req-9ce7b521-f6b8-4115-84a0-d86734b5a0b0'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:00.971083 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-9ce7b521-f6b8-4115-84a0-d86734b5a0b0 admin admin] The token cache is being invalidate because project 9aa31b0e917d4c3990ead47bc92a2254 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:04:00.971381 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-9ce7b521-f6b8-4115-84a0-d86734b5a0b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:00.972200 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-9ce7b521-f6b8-4115-84a0-d86734b5a0b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:00.973966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1369/2724] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:04:00 2022] DELETE /identity/v3/projects/9aa31b0e917d4c3990ead47bc92a2254 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:04:00.979730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db564f0a-184f-45ea-b1ed-cc64a839757a None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:00.980180 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db564f0a-184f-45ea-b1ed-cc64a839757a None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:00.983964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db564f0a-184f-45ea-b1ed-cc64a839757a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:01.005291 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-db564f0a-184f-45ea-b1ed-cc64a839757a None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:01.006643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-db564f0a-184f-45ea-b1ed-cc64a839757a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:01.032040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:01.083612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1356/2725] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:04:00 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:04:01.091271 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a9b45-03d7-4c46-be98-217806f19acb None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:01.091575 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a9b45-03d7-4c46-be98-217806f19acb None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:01.091906 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171a9b45-03d7-4c46-be98-217806f19acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:01.115613 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-171a9b45-03d7-4c46-be98-217806f19acb None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:01.116832 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-171a9b45-03d7-4c46-be98-217806f19acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:01.124734 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1370/2726] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:04:01 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:04:01.129307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db564f0a-184f-45ea-b1ed-cc64a839757a None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:01.138964 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db564f0a-184f-45ea-b1ed-cc64a839757a None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:01.143623 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6063c-122b-49c4-bf85-ae38be6f77d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:01.145219 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6063c-122b-49c4-bf85-ae38be6f77d3 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:01.145219 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6063c-122b-49c4-bf85-ae38be6f77d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:01.145420 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f6063c-122b-49c4-bf85-ae38be6f77d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:01.146032 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6063c-122b-49c4-bf85-ae38be6f77d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:04:01.146339 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6063c-122b-49c4-bf85-ae38be6f77d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:01.147980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f6063c-122b-49c4-bf85-ae38be6f77d3 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:01.159493 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1357/2727] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 11:04:01 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:04:01.164871 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a9b45-03d7-4c46-be98-217806f19acb None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:01.174246 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171a9b45-03d7-4c46-be98-217806f19acb None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:01.181157 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa43bed-6552-4097-9e32-0718e9aa70b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:01.181838 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa43bed-6552-4097-9e32-0718e9aa70b8 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:01.182118 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa43bed-6552-4097-9e32-0718e9aa70b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:01.182386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa43bed-6552-4097-9e32-0718e9aa70b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:01.183071 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa43bed-6552-4097-9e32-0718e9aa70b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:01.184978 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa43bed-6552-4097-9e32-0718e9aa70b8 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:01.275115 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1371/2728] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:04:01 2022] POST /identity/v3/projects => generated 376 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:04:01.279385 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6063c-122b-49c4-bf85-ae38be6f77d3 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:01.290119 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f6063c-122b-49c4-bf85-ae38be6f77d3 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:01.352579 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94edc94d-acbe-4cbf-9e8c-1f5458f2fef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:01.353234 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edc94d-acbe-4cbf-9e8c-1f5458f2fef6 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:01.353517 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edc94d-acbe-4cbf-9e8c-1f5458f2fef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:01.353820 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94edc94d-acbe-4cbf-9e8c-1f5458f2fef6 admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:01.354506 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94edc94d-acbe-4cbf-9e8c-1f5458f2fef6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:01.356464 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94edc94d-acbe-4cbf-9e8c-1f5458f2fef6 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:01.448827 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1358/2729] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:04:01 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:04:01.453149 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa43bed-6552-4097-9e32-0718e9aa70b8 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:01.464435 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa43bed-6552-4097-9e32-0718e9aa70b8 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:01.469986 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef91a0-ee95-4aa3-9b18-f440a24a80d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:01.470611 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef91a0-ee95-4aa3-9b18-f440a24a80d8 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:01.470893 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef91a0-ee95-4aa3-9b18-f440a24a80d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:01.471207 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef91a0-ee95-4aa3-9b18-f440a24a80d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:01.471905 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef91a0-ee95-4aa3-9b18-f440a24a80d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:01.474027 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bef91a0-ee95-4aa3-9b18-f440a24a80d8 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:01.485495 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1372/2730] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:04:01 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:04:01.490479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94edc94d-acbe-4cbf-9e8c-1f5458f2fef6 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:01.500697 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94edc94d-acbe-4cbf-9e8c-1f5458f2fef6 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:01.505973 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea92a10-eb2a-4b34-a1fb-42c22160002e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:01.506587 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea92a10-eb2a-4b34-a1fb-42c22160002e admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:01.506868 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea92a10-eb2a-4b34-a1fb-42c22160002e admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:01.507179 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea92a10-eb2a-4b34-a1fb-42c22160002e admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:01.515469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea92a10-eb2a-4b34-a1fb-42c22160002e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:01.515469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea92a10-eb2a-4b34-a1fb-42c22160002e admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:01.523982 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1359/2731] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:04:01 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:04:01.526103 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef91a0-ee95-4aa3-9b18-f440a24a80d8 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:01.537603 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef91a0-ee95-4aa3-9b18-f440a24a80d8 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:01.544190 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68019636-07cf-4d76-945a-d6b0ade2899a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:01.545040 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68019636-07cf-4d76-945a-d6b0ade2899a admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:01.545330 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68019636-07cf-4d76-945a-d6b0ade2899a admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:01.545595 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68019636-07cf-4d76-945a-d6b0ade2899a admin admin] PATH_INFO: `/v3/projects/18a92e65eb7546048be4b7c215257f70/users/9ab9773545044cbf8e6f0318fe649f9d/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:01.562301 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68019636-07cf-4d76-945a-d6b0ade2899a admin admin] RBAC: Authorizing `identity:create_grant(project_id=18a92e65eb7546048be4b7c215257f70, user_id=9ab9773545044cbf8e6f0318fe649f9d, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:01.565208 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68019636-07cf-4d76-945a-d6b0ade2899a admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:01.688629 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1373/2732] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:04:01 2022] PUT /identity/v3/projects/18a92e65eb7546048be4b7c215257f70/users/9ab9773545044cbf8e6f0318fe649f9d/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:04:01.700695 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45659916-ffc5-4c7f-8b0b-00d98b8c44e6 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:01.700981 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45659916-ffc5-4c7f-8b0b-00d98b8c44e6 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:01.701260 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45659916-ffc5-4c7f-8b0b-00d98b8c44e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:01.732875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-45659916-ffc5-4c7f-8b0b-00d98b8c44e6 None None] MFA Rules not processed for user `9ab9773545044cbf8e6f0318fe649f9d`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:01.734109 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-45659916-ffc5-4c7f-8b0b-00d98b8c44e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:01.799145 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1360/2733] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:04:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:04:01.810070 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68019636-07cf-4d76-945a-d6b0ade2899a admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:01.811832 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-68019636-07cf-4d76-945a-d6b0ade2899a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:01.834014 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:01.870426 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68019636-07cf-4d76-945a-d6b0ade2899a admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:01.875490 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6953b0-9f4f-4421-ad0f-602bfc90165e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:01.876124 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6953b0-9f4f-4421-ad0f-602bfc90165e service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:01.876397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6953b0-9f4f-4421-ad0f-602bfc90165e service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:01.876661 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6953b0-9f4f-4421-ad0f-602bfc90165e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:01.885980 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6953b0-9f4f-4421-ad0f-602bfc90165e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:01.887450 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6953b0-9f4f-4421-ad0f-602bfc90165e service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:01.897111 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1374/2734] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:04:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:04:05.689681 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea32f3-39ec-4bcd-a365-6bf8e9e1981b None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:05.695429 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea32f3-39ec-4bcd-a365-6bf8e9e1981b None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:05.695429 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eea32f3-39ec-4bcd-a365-6bf8e9e1981b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:05.709200 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-6eea32f3-39ec-4bcd-a365-6bf8e9e1981b None None] MFA Rules not processed for user `9ab9773545044cbf8e6f0318fe649f9d`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:05.710329 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-6eea32f3-39ec-4bcd-a365-6bf8e9e1981b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:05.717679 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1361/2735] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:04:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:04:06.304957 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146beee-9b1c-4b7e-a934-91cf66dbd485 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:06.305428 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146beee-9b1c-4b7e-a934-91cf66dbd485 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:06.305528 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a146beee-9b1c-4b7e-a934-91cf66dbd485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:06.345573 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-a146beee-9b1c-4b7e-a934-91cf66dbd485 None None] MFA Rules not processed for user `2b03263937cb4ca7807a24da44d37b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:06.347208 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-a146beee-9b1c-4b7e-a934-91cf66dbd485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:06.426607 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1375/2736] 10.222.0.115 () {58 vars in 1014 bytes} [Tue Aug 2 11:04:06 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:04:06.636650 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eea32f3-39ec-4bcd-a365-6bf8e9e1981b None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:06.638128 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-6eea32f3-39ec-4bcd-a365-6bf8e9e1981b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:06.657630 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:06.715353 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eea32f3-39ec-4bcd-a365-6bf8e9e1981b None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:06.721066 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd347a70-270f-4a9d-9bdc-1dfd3a82943d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0daa49b5a35f4a468bb8d0939e7ec20c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:06.727385 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd347a70-270f-4a9d-9bdc-1dfd3a82943d service swift] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:06.727385 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd347a70-270f-4a9d-9bdc-1dfd3a82943d service swift] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:06.727385 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd347a70-270f-4a9d-9bdc-1dfd3a82943d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:06.732748 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd347a70-270f-4a9d-9bdc-1dfd3a82943d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:06.734672 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd347a70-270f-4a9d-9bdc-1dfd3a82943d service swift] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:06.743978 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1362/2737] 10.222.0.115 () {60 vars in 1389 bytes} [Tue Aug 2 11:04:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 11:04:07.149895 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146beee-9b1c-4b7e-a934-91cf66dbd485 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:07.151444 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-a146beee-9b1c-4b7e-a934-91cf66dbd485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:07.232565 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a146beee-9b1c-4b7e-a934-91cf66dbd485 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:07.238320 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560804d8-9d23-40d9-88af-6861b1ac3b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:07.239747 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560804d8-9d23-40d9-88af-6861b1ac3b36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:07.240028 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560804d8-9d23-40d9-88af-6861b1ac3b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:07.240301 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560804d8-9d23-40d9-88af-6861b1ac3b36 admin admin] PATH_INFO: `/v3/users/552719fa851145b89bd2858aea9a9e03` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:07.241700 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560804d8-9d23-40d9-88af-6861b1ac3b36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=552719fa851145b89bd2858aea9a9e03)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:07.243872 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560804d8-9d23-40d9-88af-6861b1ac3b36 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:07.451617 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-560804d8-9d23-40d9-88af-6861b1ac3b36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '552719fa851145b89bd2858aea9a9e03'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:07.464989 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:07.471254 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-560804d8-9d23-40d9-88af-6861b1ac3b36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '552719fa851145b89bd2858aea9a9e03'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:07.471254 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-560804d8-9d23-40d9-88af-6861b1ac3b36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '552719fa851145b89bd2858aea9a9e03'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:07.564249 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-560804d8-9d23-40d9-88af-6861b1ac3b36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '552719fa851145b89bd2858aea9a9e03'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:07.578962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-560804d8-9d23-40d9-88af-6861b1ac3b36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '552719fa851145b89bd2858aea9a9e03'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:07.587181 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1376/2738] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:04:07 2022] DELETE /identity/v3/users/552719fa851145b89bd2858aea9a9e03 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:04:07.591618 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c38dd3-a5c0-4e53-9dfa-193377327432 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:07.591916 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c38dd3-a5c0-4e53-9dfa-193377327432 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:07.592196 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c38dd3-a5c0-4e53-9dfa-193377327432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:07.616149 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-b7c38dd3-a5c0-4e53-9dfa-193377327432 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:07.617414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-b7c38dd3-a5c0-4e53-9dfa-193377327432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:07.695141 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1363/2739] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:04:07 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:04:07.706609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560804d8-9d23-40d9-88af-6861b1ac3b36 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:07.709435 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-560804d8-9d23-40d9-88af-6861b1ac3b36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:07.782466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560804d8-9d23-40d9-88af-6861b1ac3b36 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:07.788089 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b5250-ebef-4f22-b862-2ae2753fe10a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:07.789323 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b5250-ebef-4f22-b862-2ae2753fe10a service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:07.789656 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b5250-ebef-4f22-b862-2ae2753fe10a service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:07.790140 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9b5250-ebef-4f22-b862-2ae2753fe10a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:07.809998 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9b5250-ebef-4f22-b862-2ae2753fe10a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:07.812878 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9b5250-ebef-4f22-b862-2ae2753fe10a service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:07.837373 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1377/2740] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:04:07 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:04:08.412087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c38dd3-a5c0-4e53-9dfa-193377327432 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:08.413590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-b7c38dd3-a5c0-4e53-9dfa-193377327432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:08.426802 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c38dd3-a5c0-4e53-9dfa-193377327432 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:08.435165 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407bd353-c127-486d-890b-1d9c85163384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:08.436501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407bd353-c127-486d-890b-1d9c85163384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:08.436848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407bd353-c127-486d-890b-1d9c85163384 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:08.437271 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407bd353-c127-486d-890b-1d9c85163384 admin admin] PATH_INFO: `/v3/projects/4060ad4dc3cf4855b563dc84dba99556` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:08.440470 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407bd353-c127-486d-890b-1d9c85163384 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4060ad4dc3cf4855b563dc84dba99556)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:08.444492 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407bd353-c127-486d-890b-1d9c85163384 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:08.602987 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-407bd353-c127-486d-890b-1d9c85163384 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4060ad4dc3cf4855b563dc84dba99556', 'request_id': 'req-407bd353-c127-486d-890b-1d9c85163384'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:08.613765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-407bd353-c127-486d-890b-1d9c85163384 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4060ad4dc3cf4855b563dc84dba99556', 'request_id': 'req-407bd353-c127-486d-890b-1d9c85163384'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:08.683388 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-407bd353-c127-486d-890b-1d9c85163384 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4060ad4dc3cf4855b563dc84dba99556', 'request_id': 'req-407bd353-c127-486d-890b-1d9c85163384'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:08.698658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-407bd353-c127-486d-890b-1d9c85163384 admin admin] The token cache is being invalidate because project 4060ad4dc3cf4855b563dc84dba99556 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:04:08.698998 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-407bd353-c127-486d-890b-1d9c85163384 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:08.700077 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-407bd353-c127-486d-890b-1d9c85163384 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:08.702149 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1364/2741] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:04:08 2022] DELETE /identity/v3/projects/4060ad4dc3cf4855b563dc84dba99556 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:04:08.706503 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b5250-ebef-4f22-b862-2ae2753fe10a service neutron] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:08.718930 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9b5250-ebef-4f22-b862-2ae2753fe10a service neutron] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:08.799172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcee489-f938-4ce6-8b42-48c982055856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:08.799576 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcee489-f938-4ce6-8b42-48c982055856 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:08.799873 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcee489-f938-4ce6-8b42-48c982055856 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:08.800194 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcee489-f938-4ce6-8b42-48c982055856 admin admin] PATH_INFO: `/v3/users/e8d8024c428b4de4ae95219511a2be12` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:08.801745 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcee489-f938-4ce6-8b42-48c982055856 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8d8024c428b4de4ae95219511a2be12)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:08.803994 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcee489-f938-4ce6-8b42-48c982055856 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:09.167432 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2dcee489-f938-4ce6-8b42-48c982055856 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8d8024c428b4de4ae95219511a2be12'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:09.181381 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2dcee489-f938-4ce6-8b42-48c982055856 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8d8024c428b4de4ae95219511a2be12'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:09.182161 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2dcee489-f938-4ce6-8b42-48c982055856 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8d8024c428b4de4ae95219511a2be12'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:09.279292 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2dcee489-f938-4ce6-8b42-48c982055856 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8d8024c428b4de4ae95219511a2be12'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:09.308217 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:09.310437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2dcee489-f938-4ce6-8b42-48c982055856 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8d8024c428b4de4ae95219511a2be12'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:09.322993 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1378/2742] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:04:08 2022] DELETE /identity/v3/users/e8d8024c428b4de4ae95219511a2be12 => generated 0 bytes in 616 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:04:10.278694 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407bd353-c127-486d-890b-1d9c85163384 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:10.282093 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-407bd353-c127-486d-890b-1d9c85163384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:10.306660 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407bd353-c127-486d-890b-1d9c85163384 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:10.331272 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:10.384759 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ff97c7-9a4d-4bea-b556-b4e5ea6cdcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:10.385586 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff97c7-9a4d-4bea-b556-b4e5ea6cdcb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:10.385877 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff97c7-9a4d-4bea-b556-b4e5ea6cdcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:10.386162 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ff97c7-9a4d-4bea-b556-b4e5ea6cdcb5 admin admin] PATH_INFO: `/v3/projects/b06e07371fdd43f480c7c29753d600a8` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:10.387430 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ff97c7-9a4d-4bea-b556-b4e5ea6cdcb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b06e07371fdd43f480c7c29753d600a8)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:10.389352 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ff97c7-9a4d-4bea-b556-b4e5ea6cdcb5 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:10.541483 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-18ff97c7-9a4d-4bea-b556-b4e5ea6cdcb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b06e07371fdd43f480c7c29753d600a8', 'request_id': 'req-18ff97c7-9a4d-4bea-b556-b4e5ea6cdcb5'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:10.569121 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-18ff97c7-9a4d-4bea-b556-b4e5ea6cdcb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b06e07371fdd43f480c7c29753d600a8', 'request_id': 'req-18ff97c7-9a4d-4bea-b556-b4e5ea6cdcb5'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:10.650479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-18ff97c7-9a4d-4bea-b556-b4e5ea6cdcb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b06e07371fdd43f480c7c29753d600a8', 'request_id': 'req-18ff97c7-9a4d-4bea-b556-b4e5ea6cdcb5'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:10.665139 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-18ff97c7-9a4d-4bea-b556-b4e5ea6cdcb5 admin admin] The token cache is being invalidate because project b06e07371fdd43f480c7c29753d600a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:04:10.665821 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-18ff97c7-9a4d-4bea-b556-b4e5ea6cdcb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:10.666720 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-18ff97c7-9a4d-4bea-b556-b4e5ea6cdcb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:10.668972 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1365/2743] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:04:10 2022] DELETE /identity/v3/projects/b06e07371fdd43f480c7c29753d600a8 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:04:10.673995 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac40ce9-4e6d-49ea-9f92-34ffa0e83928 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:10.674509 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac40ce9-4e6d-49ea-9f92-34ffa0e83928 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:10.676592 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac40ce9-4e6d-49ea-9f92-34ffa0e83928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:10.699236 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-eac40ce9-4e6d-49ea-9f92-34ffa0e83928 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:10.700584 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-eac40ce9-4e6d-49ea-9f92-34ffa0e83928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:10.815287 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1379/2744] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:04:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:04:10.824396 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f79402-b024-4195-82ec-92b245b3ded8 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:10.825006 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f79402-b024-4195-82ec-92b245b3ded8 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:10.825512 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f79402-b024-4195-82ec-92b245b3ded8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:10.868145 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-c0f79402-b024-4195-82ec-92b245b3ded8 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:10.871562 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-c0f79402-b024-4195-82ec-92b245b3ded8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:10.880848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1366/2745] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:04:10 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:04:10.884854 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac40ce9-4e6d-49ea-9f92-34ffa0e83928 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:10.896077 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac40ce9-4e6d-49ea-9f92-34ffa0e83928 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:10.901901 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea73c0f8-6eba-4fd2-b4c3-a516c6f74917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:10.902465 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73c0f8-6eba-4fd2-b4c3-a516c6f74917 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:10.902738 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73c0f8-6eba-4fd2-b4c3-a516c6f74917 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:10.903031 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea73c0f8-6eba-4fd2-b4c3-a516c6f74917 admin admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:10.903761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea73c0f8-6eba-4fd2-b4c3-a516c6f74917 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:04:10.904067 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea73c0f8-6eba-4fd2-b4c3-a516c6f74917 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:10.905681 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea73c0f8-6eba-4fd2-b4c3-a516c6f74917 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:10.924314 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1380/2746] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 11:04:10 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:04:10.926869 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f79402-b024-4195-82ec-92b245b3ded8 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:10.940228 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f79402-b024-4195-82ec-92b245b3ded8 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:10.947358 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d54d7-ddd4-4490-82ee-c92c496c226b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:10.948401 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d54d7-ddd4-4490-82ee-c92c496c226b admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:10.949056 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d54d7-ddd4-4490-82ee-c92c496c226b admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:10.949705 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d54d7-ddd4-4490-82ee-c92c496c226b admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:10.951421 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76d54d7-ddd4-4490-82ee-c92c496c226b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:10.954799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76d54d7-ddd4-4490-82ee-c92c496c226b admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:11.054688 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1367/2747] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:04:10 2022] POST /identity/v3/projects => generated 364 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:04:11.057605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea73c0f8-6eba-4fd2-b4c3-a516c6f74917 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:11.072708 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea73c0f8-6eba-4fd2-b4c3-a516c6f74917 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:11.137263 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b970a-3cdc-4542-86d1-68c894bc1412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:11.137977 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b970a-3cdc-4542-86d1-68c894bc1412 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:11.138329 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b970a-3cdc-4542-86d1-68c894bc1412 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:11.138671 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b970a-3cdc-4542-86d1-68c894bc1412 admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:11.139453 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588b970a-3cdc-4542-86d1-68c894bc1412 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:11.141456 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588b970a-3cdc-4542-86d1-68c894bc1412 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:11.220954 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1381/2748] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:04:11 2022] POST /identity/v3/users => generated 283 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:04:11.225446 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d54d7-ddd4-4490-82ee-c92c496c226b admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:11.237279 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76d54d7-ddd4-4490-82ee-c92c496c226b admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:11.243485 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d04b8-8125-4bea-8a90-3e9ee844fe76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:11.244261 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d04b8-8125-4bea-8a90-3e9ee844fe76 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:11.244688 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d04b8-8125-4bea-8a90-3e9ee844fe76 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:11.245498 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d04b8-8125-4bea-8a90-3e9ee844fe76 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:11.246028 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d04b8-8125-4bea-8a90-3e9ee844fe76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:11.247758 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d04b8-8125-4bea-8a90-3e9ee844fe76 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:11.259811 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1368/2749] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:04:11 2022] GET /identity/v3/roles => generated 1427 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:04:11.263889 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b970a-3cdc-4542-86d1-68c894bc1412 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:11.275952 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b970a-3cdc-4542-86d1-68c894bc1412 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:11.281212 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdc2306-f34c-4e70-aefd-fa74c63cf26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:11.281821 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc2306-f34c-4e70-aefd-fa74c63cf26d admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:11.282163 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc2306-f34c-4e70-aefd-fa74c63cf26d admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:11.282382 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdc2306-f34c-4e70-aefd-fa74c63cf26d admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:11.283273 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdc2306-f34c-4e70-aefd-fa74c63cf26d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:11.284875 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffdc2306-f34c-4e70-aefd-fa74c63cf26d admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:11.303095 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1382/2750] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:04:11 2022] GET /identity/v3/roles => generated 1427 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:04:11.306881 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d04b8-8125-4bea-8a90-3e9ee844fe76 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:11.319130 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d04b8-8125-4bea-8a90-3e9ee844fe76 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:11.324100 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d78a0-610b-471d-8ce7-cbcfe7c44652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:11.324769 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d78a0-610b-471d-8ce7-cbcfe7c44652 admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:11.325052 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d78a0-610b-471d-8ce7-cbcfe7c44652 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:11.325332 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788d78a0-610b-471d-8ce7-cbcfe7c44652 admin admin] PATH_INFO: `/v3/projects/1f96b2c5aa17445e9084200b268eefaf/users/100a9ca8b6564708ad9314a912ecdc28/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:11.341551 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d78a0-610b-471d-8ce7-cbcfe7c44652 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f96b2c5aa17445e9084200b268eefaf, user_id=100a9ca8b6564708ad9314a912ecdc28, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:11.344292 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788d78a0-610b-471d-8ce7-cbcfe7c44652 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:11.437170 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1369/2751] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:04:11 2022] PUT /identity/v3/projects/1f96b2c5aa17445e9084200b268eefaf/users/100a9ca8b6564708ad9314a912ecdc28/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:04:11.447614 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d067aa5-65d8-44e4-b0ac-ffebc586060c None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:11.447900 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d067aa5-65d8-44e4-b0ac-ffebc586060c None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:11.448199 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d067aa5-65d8-44e4-b0ac-ffebc586060c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:11.478936 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-8d067aa5-65d8-44e4-b0ac-ffebc586060c None None] MFA Rules not processed for user `100a9ca8b6564708ad9314a912ecdc28`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:11.480067 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-8d067aa5-65d8-44e4-b0ac-ffebc586060c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:11.535439 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:11.555993 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1383/2752] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:04:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:04:11.560606 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d78a0-610b-471d-8ce7-cbcfe7c44652 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:11.640575 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:11.680995 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788d78a0-610b-471d-8ce7-cbcfe7c44652 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:11.680995 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d8d49-32f9-4ed0-a908-38071754ea9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:11.680995 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d8d49-32f9-4ed0-a908-38071754ea9c admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:11.680995 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d8d49-32f9-4ed0-a908-38071754ea9c admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:11.681539 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8d8d49-32f9-4ed0-a908-38071754ea9c admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:11.681539 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8d8d49-32f9-4ed0-a908-38071754ea9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:11.681539 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8d8d49-32f9-4ed0-a908-38071754ea9c admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:11.803088 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1370/2753] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:04:11 2022] POST /identity/v3/projects => generated 370 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:04:11.809496 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d067aa5-65d8-44e4-b0ac-ffebc586060c None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:11.821307 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d067aa5-65d8-44e4-b0ac-ffebc586060c None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:11.886596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17498676-c45b-4f5e-a03c-808b892c1e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:11.887337 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17498676-c45b-4f5e-a03c-808b892c1e1f admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:11.887614 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17498676-c45b-4f5e-a03c-808b892c1e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:11.887884 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17498676-c45b-4f5e-a03c-808b892c1e1f admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:11.888544 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17498676-c45b-4f5e-a03c-808b892c1e1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:11.890442 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17498676-c45b-4f5e-a03c-808b892c1e1f admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:11.971769 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1384/2754] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:04:11 2022] POST /identity/v3/users => generated 292 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:04:11.975997 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d8d49-32f9-4ed0-a908-38071754ea9c admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:12.012386 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8d8d49-32f9-4ed0-a908-38071754ea9c admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:12.018113 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366c80d-5e6c-4810-b1a8-f0dd9ca8d797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:12.018771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366c80d-5e6c-4810-b1a8-f0dd9ca8d797 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:12.021202 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366c80d-5e6c-4810-b1a8-f0dd9ca8d797 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:12.021202 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1366c80d-5e6c-4810-b1a8-f0dd9ca8d797 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:12.021493 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366c80d-5e6c-4810-b1a8-f0dd9ca8d797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:12.023119 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1366c80d-5e6c-4810-b1a8-f0dd9ca8d797 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:12.051524 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1371/2755] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:04:11 2022] GET /identity/v3/roles => generated 1427 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:04:12.052897 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17498676-c45b-4f5e-a03c-808b892c1e1f admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:12.064994 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17498676-c45b-4f5e-a03c-808b892c1e1f admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:12.076156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7314f7ba-9ca8-4c17-b73c-f11cd402bce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:12.077238 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314f7ba-9ca8-4c17-b73c-f11cd402bce1 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:12.078130 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314f7ba-9ca8-4c17-b73c-f11cd402bce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:12.078316 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7314f7ba-9ca8-4c17-b73c-f11cd402bce1 admin admin] PATH_INFO: `/v3/projects/f7e8542deee245b68808c4e2fc766b8e/users/92271ada8e454e1aa963bcde31ae510b/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:12.104359 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7314f7ba-9ca8-4c17-b73c-f11cd402bce1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7e8542deee245b68808c4e2fc766b8e, user_id=92271ada8e454e1aa963bcde31ae510b, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:12.107188 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7314f7ba-9ca8-4c17-b73c-f11cd402bce1 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:12.467698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1385/2756] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:04:12 2022] PUT /identity/v3/projects/f7e8542deee245b68808c4e2fc766b8e/users/92271ada8e454e1aa963bcde31ae510b/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 415 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:04:12.471782 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366c80d-5e6c-4810-b1a8-f0dd9ca8d797 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:12.482082 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1366c80d-5e6c-4810-b1a8-f0dd9ca8d797 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:12.576341 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dde828-6191-4afc-b0d9-c0034a4c66b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:12.577127 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dde828-6191-4afc-b0d9-c0034a4c66b0 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:12.577513 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dde828-6191-4afc-b0d9-c0034a4c66b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:12.577902 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dde828-6191-4afc-b0d9-c0034a4c66b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:12.578841 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dde828-6191-4afc-b0d9-c0034a4c66b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:12.580950 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dde828-6191-4afc-b0d9-c0034a4c66b0 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:12.597150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1372/2757] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:04:12 2022] GET /identity/v3/roles => generated 1427 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:04:12.598586 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7314f7ba-9ca8-4c17-b73c-f11cd402bce1 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:12.617038 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7314f7ba-9ca8-4c17-b73c-f11cd402bce1 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:12.630963 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dca94d-fe75-4a79-975a-6737e835b4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:12.631580 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dca94d-fe75-4a79-975a-6737e835b4e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:12.631863 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dca94d-fe75-4a79-975a-6737e835b4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:12.632135 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54dca94d-fe75-4a79-975a-6737e835b4e0 admin admin] PATH_INFO: `/v3/projects/f7e8542deee245b68808c4e2fc766b8e/users/92271ada8e454e1aa963bcde31ae510b/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:12.648271 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dca94d-fe75-4a79-975a-6737e835b4e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7e8542deee245b68808c4e2fc766b8e, user_id=92271ada8e454e1aa963bcde31ae510b, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:12.651167 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54dca94d-fe75-4a79-975a-6737e835b4e0 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:12.678426 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1386/2758] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:04:12 2022] PUT /identity/v3/projects/f7e8542deee245b68808c4e2fc766b8e/users/92271ada8e454e1aa963bcde31ae510b/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:04:12.692653 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb1033-c53c-4a78-854a-bebf74a2a5e6 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:12.692971 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb1033-c53c-4a78-854a-bebf74a2a5e6 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:12.693206 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcb1033-c53c-4a78-854a-bebf74a2a5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:12.746263 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-9fcb1033-c53c-4a78-854a-bebf74a2a5e6 None None] MFA Rules not processed for user `92271ada8e454e1aa963bcde31ae510b`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:12.748046 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-9fcb1033-c53c-4a78-854a-bebf74a2a5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:12.865016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1373/2759] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:04:12 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:04:12.876730 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dca94d-fe75-4a79-975a-6737e835b4e0 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:12.878568 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-54dca94d-fe75-4a79-975a-6737e835b4e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:12.952613 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54dca94d-fe75-4a79-975a-6737e835b4e0 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:12.957564 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea299185-15dc-4f94-9b10-67beaaa688ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:12.958357 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea299185-15dc-4f94-9b10-67beaaa688ef service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:12.959227 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea299185-15dc-4f94-9b10-67beaaa688ef service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:12.959227 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea299185-15dc-4f94-9b10-67beaaa688ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:12.969426 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea299185-15dc-4f94-9b10-67beaaa688ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:12.971029 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea299185-15dc-4f94-9b10-67beaaa688ef service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:12.983026 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1387/2760] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:04:12 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:04:13.126296 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb1033-c53c-4a78-854a-bebf74a2a5e6 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:13.127898 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-9fcb1033-c53c-4a78-854a-bebf74a2a5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:13.144848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcb1033-c53c-4a78-854a-bebf74a2a5e6 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:13.149906 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1ee86-6c6a-43b7-94d9-91a941af40e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:13.150676 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1ee86-6c6a-43b7-94d9-91a941af40e4 service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:13.151206 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1ee86-6c6a-43b7-94d9-91a941af40e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:13.151734 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1ee86-6c6a-43b7-94d9-91a941af40e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:13.216285 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:13.234118 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa1ee86-6c6a-43b7-94d9-91a941af40e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:13.235654 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa1ee86-6c6a-43b7-94d9-91a941af40e4 service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:13.246579 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1374/2761] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:04:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:04:18.650823 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea299185-15dc-4f94-9b10-67beaaa688ef service cinder] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:18.713689 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:18.737840 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea299185-15dc-4f94-9b10-67beaaa688ef service cinder] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:18.742948 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce689fa9-d254-464e-a54e-7ad9037f9c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:18.743880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce689fa9-d254-464e-a54e-7ad9037f9c9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:18.744278 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce689fa9-d254-464e-a54e-7ad9037f9c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:18.744617 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce689fa9-d254-464e-a54e-7ad9037f9c9e admin admin] PATH_INFO: `/v3/users/100a9ca8b6564708ad9314a912ecdc28` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:18.745861 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce689fa9-d254-464e-a54e-7ad9037f9c9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=100a9ca8b6564708ad9314a912ecdc28)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:18.747892 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce689fa9-d254-464e-a54e-7ad9037f9c9e admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:19.029065 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ce689fa9-d254-464e-a54e-7ad9037f9c9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '100a9ca8b6564708ad9314a912ecdc28'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:19.040896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ce689fa9-d254-464e-a54e-7ad9037f9c9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '100a9ca8b6564708ad9314a912ecdc28'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:19.041710 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ce689fa9-d254-464e-a54e-7ad9037f9c9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '100a9ca8b6564708ad9314a912ecdc28'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:19.107967 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ce689fa9-d254-464e-a54e-7ad9037f9c9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '100a9ca8b6564708ad9314a912ecdc28'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:19.137052 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-ce689fa9-d254-464e-a54e-7ad9037f9c9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '100a9ca8b6564708ad9314a912ecdc28'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:19.146131 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1388/2762] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:04:18 2022] DELETE /identity/v3/users/100a9ca8b6564708ad9314a912ecdc28 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:04:19.150971 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fa6ca-d439-4aa4-bc59-5ee0b7c4bac7 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:19.151551 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fa6ca-d439-4aa4-bc59-5ee0b7c4bac7 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:19.151551 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fa6ca-d439-4aa4-bc59-5ee0b7c4bac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:19.188996 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-0c7fa6ca-d439-4aa4-bc59-5ee0b7c4bac7 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:19.190184 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-0c7fa6ca-d439-4aa4-bc59-5ee0b7c4bac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:19.275315 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1375/2763] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:04:19 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:04:19.286769 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce689fa9-d254-464e-a54e-7ad9037f9c9e admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:19.288531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-ce689fa9-d254-464e-a54e-7ad9037f9c9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:19.368485 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce689fa9-d254-464e-a54e-7ad9037f9c9e admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:19.372742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f8adf-c956-4dc4-8954-4a79ef1006d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:19.373381 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f8adf-c956-4dc4-8954-4a79ef1006d7 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:19.373910 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f8adf-c956-4dc4-8954-4a79ef1006d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:19.374090 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765f8adf-c956-4dc4-8954-4a79ef1006d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:19.389743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f8adf-c956-4dc4-8954-4a79ef1006d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:19.392974 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765f8adf-c956-4dc4-8954-4a79ef1006d7 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:19.406572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1389/2764] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:04:19 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:04:19.944066 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7fa6ca-d439-4aa4-bc59-5ee0b7c4bac7 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:19.945604 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-0c7fa6ca-d439-4aa4-bc59-5ee0b7c4bac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:19.963475 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7fa6ca-d439-4aa4-bc59-5ee0b7c4bac7 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:19.970138 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5f072-92e9-4ff0-86af-9823cb6f64ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:19.970846 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5f072-92e9-4ff0-86af-9823cb6f64ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:19.971172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5f072-92e9-4ff0-86af-9823cb6f64ad admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:19.971445 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe5f072-92e9-4ff0-86af-9823cb6f64ad admin admin] PATH_INFO: `/v3/projects/1f96b2c5aa17445e9084200b268eefaf` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:19.973461 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5f072-92e9-4ff0-86af-9823cb6f64ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f96b2c5aa17445e9084200b268eefaf)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:19.975894 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe5f072-92e9-4ff0-86af-9823cb6f64ad admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:20.111711 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-3fe5f072-92e9-4ff0-86af-9823cb6f64ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f96b2c5aa17445e9084200b268eefaf', 'request_id': 'req-3fe5f072-92e9-4ff0-86af-9823cb6f64ad'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:20.123379 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-3fe5f072-92e9-4ff0-86af-9823cb6f64ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f96b2c5aa17445e9084200b268eefaf', 'request_id': 'req-3fe5f072-92e9-4ff0-86af-9823cb6f64ad'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:20.203633 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-3fe5f072-92e9-4ff0-86af-9823cb6f64ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f96b2c5aa17445e9084200b268eefaf', 'request_id': 'req-3fe5f072-92e9-4ff0-86af-9823cb6f64ad'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:20.246515 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-3fe5f072-92e9-4ff0-86af-9823cb6f64ad admin admin] The token cache is being invalidate because project 1f96b2c5aa17445e9084200b268eefaf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:04:20.246515 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-3fe5f072-92e9-4ff0-86af-9823cb6f64ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:20.246515 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-3fe5f072-92e9-4ff0-86af-9823cb6f64ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:20.246515 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1376/2765] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:04:19 2022] DELETE /identity/v3/projects/1f96b2c5aa17445e9084200b268eefaf => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:04:20.248044 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f8adf-c956-4dc4-8954-4a79ef1006d7 service neutron] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:20.281082 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765f8adf-c956-4dc4-8954-4a79ef1006d7 service neutron] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:20.533884 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8d8c7-25b8-4625-a609-7d8c0c7e1d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:20.533884 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8d8c7-25b8-4625-a609-7d8c0c7e1d9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:20.533884 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8d8c7-25b8-4625-a609-7d8c0c7e1d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:20.533884 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8d8c7-25b8-4625-a609-7d8c0c7e1d9c admin admin] PATH_INFO: `/v3/users/92271ada8e454e1aa963bcde31ae510b` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:20.536951 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef8d8c7-25b8-4625-a609-7d8c0c7e1d9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=92271ada8e454e1aa963bcde31ae510b)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:20.536951 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef8d8c7-25b8-4625-a609-7d8c0c7e1d9c admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:20.536951 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:20.780497 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-4ef8d8c7-25b8-4625-a609-7d8c0c7e1d9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92271ada8e454e1aa963bcde31ae510b'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:20.804999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-4ef8d8c7-25b8-4625-a609-7d8c0c7e1d9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92271ada8e454e1aa963bcde31ae510b'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:20.806062 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-4ef8d8c7-25b8-4625-a609-7d8c0c7e1d9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92271ada8e454e1aa963bcde31ae510b'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:20.893249 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-4ef8d8c7-25b8-4625-a609-7d8c0c7e1d9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92271ada8e454e1aa963bcde31ae510b'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:20.934580 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-4ef8d8c7-25b8-4625-a609-7d8c0c7e1d9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92271ada8e454e1aa963bcde31ae510b'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:20.952132 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1390/2766] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:04:20 2022] DELETE /identity/v3/users/92271ada8e454e1aa963bcde31ae510b => generated 0 bytes in 705 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:04:21.753850 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5f072-92e9-4ff0-86af-9823cb6f64ad admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:21.756429 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-3fe5f072-92e9-4ff0-86af-9823cb6f64ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:21.774287 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe5f072-92e9-4ff0-86af-9823cb6f64ad admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:21.879612 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33747eb9-9989-47f3-9feb-700d81547155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:21.880381 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33747eb9-9989-47f3-9feb-700d81547155 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:21.880614 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33747eb9-9989-47f3-9feb-700d81547155 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:21.880857 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33747eb9-9989-47f3-9feb-700d81547155 admin admin] PATH_INFO: `/v3/projects/f7e8542deee245b68808c4e2fc766b8e` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:21.882245 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33747eb9-9989-47f3-9feb-700d81547155 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7e8542deee245b68808c4e2fc766b8e)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:21.884238 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33747eb9-9989-47f3-9feb-700d81547155 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:21.897007 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:22.044602 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-33747eb9-9989-47f3-9feb-700d81547155 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7e8542deee245b68808c4e2fc766b8e', 'request_id': 'req-33747eb9-9989-47f3-9feb-700d81547155'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:22.051689 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-33747eb9-9989-47f3-9feb-700d81547155 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7e8542deee245b68808c4e2fc766b8e', 'request_id': 'req-33747eb9-9989-47f3-9feb-700d81547155'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:22.110339 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-33747eb9-9989-47f3-9feb-700d81547155 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7e8542deee245b68808c4e2fc766b8e', 'request_id': 'req-33747eb9-9989-47f3-9feb-700d81547155'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:22.122894 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-33747eb9-9989-47f3-9feb-700d81547155 admin admin] The token cache is being invalidate because project f7e8542deee245b68808c4e2fc766b8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:04:22.123360 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-33747eb9-9989-47f3-9feb-700d81547155 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:22.124150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-33747eb9-9989-47f3-9feb-700d81547155 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:22.125853 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1377/2767] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:04:21 2022] DELETE /identity/v3/projects/f7e8542deee245b68808c4e2fc766b8e => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:04:22.131409 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db7544-edb8-4275-800a-11ec02bcd8ff None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:22.132325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db7544-edb8-4275-800a-11ec02bcd8ff None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:22.132799 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5db7544-edb8-4275-800a-11ec02bcd8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:22.159303 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-c5db7544-edb8-4275-800a-11ec02bcd8ff None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:22.160062 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-c5db7544-edb8-4275-800a-11ec02bcd8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:22.248327 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1391/2768] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:04:22 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:04:22.257816 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d3edd-f4ef-4e72-8c75-8e04bb1ee7dd None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:22.259189 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d3edd-f4ef-4e72-8c75-8e04bb1ee7dd None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:22.259697 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156d3edd-f4ef-4e72-8c75-8e04bb1ee7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:22.303345 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-156d3edd-f4ef-4e72-8c75-8e04bb1ee7dd None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:22.304816 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-156d3edd-f4ef-4e72-8c75-8e04bb1ee7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:22.329153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1378/2769] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:04:22 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:04:22.330141 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db7544-edb8-4275-800a-11ec02bcd8ff None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:22.343543 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5db7544-edb8-4275-800a-11ec02bcd8ff None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:22.369870 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b890e-3cd7-4241-95e4-401d40b55828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:22.370470 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b890e-3cd7-4241-95e4-401d40b55828 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:22.370761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b890e-3cd7-4241-95e4-401d40b55828 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:22.371039 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370b890e-3cd7-4241-95e4-401d40b55828 admin admin] PATH_INFO: `/v3/domains` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:22.371962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b890e-3cd7-4241-95e4-401d40b55828 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92111) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:04:22.372353 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b890e-3cd7-4241-95e4-401d40b55828 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:22.374241 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370b890e-3cd7-4241-95e4-401d40b55828 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:22.404234 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1392/2770] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 11:04:22 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 11:04:22.408634 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d3edd-f4ef-4e72-8c75-8e04bb1ee7dd None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:22.420058 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156d3edd-f4ef-4e72-8c75-8e04bb1ee7dd None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:22.426766 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0843c5-4168-4f8d-af91-9fefef133b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:22.427420 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0843c5-4168-4f8d-af91-9fefef133b3f admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:22.427707 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0843c5-4168-4f8d-af91-9fefef133b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:22.427991 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0843c5-4168-4f8d-af91-9fefef133b3f admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:22.428695 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0843c5-4168-4f8d-af91-9fefef133b3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:22.430735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0843c5-4168-4f8d-af91-9fefef133b3f admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:22.554877 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1379/2771] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:04:22 2022] POST /identity/v3/projects => generated 382 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:04:22.555659 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b890e-3cd7-4241-95e4-401d40b55828 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:22.567770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370b890e-3cd7-4241-95e4-401d40b55828 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:22.666056 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d665c-63fa-42d6-8033-af2ea58a7af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:22.666913 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d665c-63fa-42d6-8033-af2ea58a7af8 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:22.667332 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d665c-63fa-42d6-8033-af2ea58a7af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:22.667609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2d665c-63fa-42d6-8033-af2ea58a7af8 admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:22.668423 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2d665c-63fa-42d6-8033-af2ea58a7af8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:22.670438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2d665c-63fa-42d6-8033-af2ea58a7af8 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:22.759578 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1393/2772] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:04:22 2022] POST /identity/v3/users => generated 292 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:04:22.760590 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0843c5-4168-4f8d-af91-9fefef133b3f admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:22.772146 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0843c5-4168-4f8d-af91-9fefef133b3f admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:22.776856 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40556f5-df72-44b5-94dc-8baa555b5ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:22.777469 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40556f5-df72-44b5-94dc-8baa555b5ff9 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:22.777754 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40556f5-df72-44b5-94dc-8baa555b5ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:22.778035 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40556f5-df72-44b5-94dc-8baa555b5ff9 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:22.778753 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40556f5-df72-44b5-94dc-8baa555b5ff9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:22.780386 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40556f5-df72-44b5-94dc-8baa555b5ff9 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:22.792332 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1380/2773] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:04:22 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:04:22.796540 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d665c-63fa-42d6-8033-af2ea58a7af8 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:22.802789 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:22.810416 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2d665c-63fa-42d6-8033-af2ea58a7af8 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:22.816659 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e89040d-88c0-41fa-aa65-c3a421ee68f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:22.821450 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89040d-88c0-41fa-aa65-c3a421ee68f6 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:22.821450 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89040d-88c0-41fa-aa65-c3a421ee68f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:22.821450 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e89040d-88c0-41fa-aa65-c3a421ee68f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:22.821450 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e89040d-88c0-41fa-aa65-c3a421ee68f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:22.821450 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e89040d-88c0-41fa-aa65-c3a421ee68f6 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:22.839643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1394/2774] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:04:22 2022] GET /identity/v3/roles => generated 1427 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:04:22.841236 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40556f5-df72-44b5-94dc-8baa555b5ff9 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:22.853374 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40556f5-df72-44b5-94dc-8baa555b5ff9 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:22.864501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b37ee5-f3fe-4113-a7ae-dc501ad88352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:22.865378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b37ee5-f3fe-4113-a7ae-dc501ad88352 admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:22.865802 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b37ee5-f3fe-4113-a7ae-dc501ad88352 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:22.866332 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b37ee5-f3fe-4113-a7ae-dc501ad88352 admin admin] PATH_INFO: `/v3/projects/29e31e4614f945dea9d528e9e91b673c/users/1b341bf52b50475bbd6a2d7792bcb92e/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:22.889239 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b37ee5-f3fe-4113-a7ae-dc501ad88352 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29e31e4614f945dea9d528e9e91b673c, user_id=1b341bf52b50475bbd6a2d7792bcb92e, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:22.892594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b37ee5-f3fe-4113-a7ae-dc501ad88352 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:22.988130 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1381/2775] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:04:22 2022] PUT /identity/v3/projects/29e31e4614f945dea9d528e9e91b673c/users/1b341bf52b50475bbd6a2d7792bcb92e/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:04:22.998592 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6741e8e-5ad7-4275-92f1-681eef9aa716 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:22.998928 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6741e8e-5ad7-4275-92f1-681eef9aa716 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:22.999253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6741e8e-5ad7-4275-92f1-681eef9aa716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:23.032345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-c6741e8e-5ad7-4275-92f1-681eef9aa716 None None] MFA Rules not processed for user `1b341bf52b50475bbd6a2d7792bcb92e`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:23.033577 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-c6741e8e-5ad7-4275-92f1-681eef9aa716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:23.131789 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1395/2776] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:04:22 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:04:23.136320 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b37ee5-f3fe-4113-a7ae-dc501ad88352 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:23.214767 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b37ee5-f3fe-4113-a7ae-dc501ad88352 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:23.221220 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4180ee52-5682-48ed-8575-cfb0f1eebfaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:23.221986 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180ee52-5682-48ed-8575-cfb0f1eebfaa admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:23.222538 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180ee52-5682-48ed-8575-cfb0f1eebfaa admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:23.222888 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4180ee52-5682-48ed-8575-cfb0f1eebfaa admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:23.223645 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4180ee52-5682-48ed-8575-cfb0f1eebfaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:23.225627 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4180ee52-5682-48ed-8575-cfb0f1eebfaa admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:23.347582 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1382/2777] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:04:23 2022] POST /identity/v3/projects => generated 380 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:04:23.349221 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6741e8e-5ad7-4275-92f1-681eef9aa716 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:23.361722 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6741e8e-5ad7-4275-92f1-681eef9aa716 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:23.428864 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accceed3-e966-4251-8887-d11600db9217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:23.429489 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accceed3-e966-4251-8887-d11600db9217 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:23.430022 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accceed3-e966-4251-8887-d11600db9217 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:23.430231 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accceed3-e966-4251-8887-d11600db9217 admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:23.431421 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accceed3-e966-4251-8887-d11600db9217 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:23.434004 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accceed3-e966-4251-8887-d11600db9217 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:23.644628 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1396/2778] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:04:23 2022] POST /identity/v3/users => generated 297 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:04:23.648944 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4180ee52-5682-48ed-8575-cfb0f1eebfaa admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:23.677948 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4180ee52-5682-48ed-8575-cfb0f1eebfaa admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:23.692857 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0a313-3b74-4972-806e-8a064cb7fbaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:23.692857 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0a313-3b74-4972-806e-8a064cb7fbaa admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:23.692857 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0a313-3b74-4972-806e-8a064cb7fbaa admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:23.692857 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0a313-3b74-4972-806e-8a064cb7fbaa admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:23.697948 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f0a313-3b74-4972-806e-8a064cb7fbaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:23.697948 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f0a313-3b74-4972-806e-8a064cb7fbaa admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:23.710859 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1383/2779] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:04:23 2022] GET /identity/v3/roles => generated 1427 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:04:23.718698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accceed3-e966-4251-8887-d11600db9217 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:23.733956 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accceed3-e966-4251-8887-d11600db9217 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:23.747550 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd8e91e-5438-450b-98a0-ef3cbe59fc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:23.749771 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8e91e-5438-450b-98a0-ef3cbe59fc7a admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:23.750371 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8e91e-5438-450b-98a0-ef3cbe59fc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:23.750849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8e91e-5438-450b-98a0-ef3cbe59fc7a admin admin] PATH_INFO: `/v3/projects/f79c5b00a8164dec8db277f492af236d/users/b54a8aed950749b78263986ca93511d4/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:23.776753 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd8e91e-5438-450b-98a0-ef3cbe59fc7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f79c5b00a8164dec8db277f492af236d, user_id=b54a8aed950749b78263986ca93511d4, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:23.783280 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd8e91e-5438-450b-98a0-ef3cbe59fc7a admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:23.891285 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1397/2780] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:04:23 2022] PUT /identity/v3/projects/f79c5b00a8164dec8db277f492af236d/users/b54a8aed950749b78263986ca93511d4/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:04:23.896538 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0a313-3b74-4972-806e-8a064cb7fbaa admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:23.906484 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0a313-3b74-4972-806e-8a064cb7fbaa admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:23.911691 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:23.985642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddc03da-c3ca-48f3-b625-33107ac087b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:24.091900 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc03da-c3ca-48f3-b625-33107ac087b8 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:24.091900 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc03da-c3ca-48f3-b625-33107ac087b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:24.091900 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddc03da-c3ca-48f3-b625-33107ac087b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:24.091900 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddc03da-c3ca-48f3-b625-33107ac087b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:24.091900 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddc03da-c3ca-48f3-b625-33107ac087b8 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:24.091900 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1384/2781] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:04:23 2022] GET /identity/v3/roles => generated 1427 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:04:24.095041 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd8e91e-5438-450b-98a0-ef3cbe59fc7a admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:24.106191 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd8e91e-5438-450b-98a0-ef3cbe59fc7a admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:24.111346 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a8b44-2c1c-4961-a1ff-c26cea04968a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:24.113657 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a8b44-2c1c-4961-a1ff-c26cea04968a admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:24.113657 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a8b44-2c1c-4961-a1ff-c26cea04968a admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:24.113657 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8a8b44-2c1c-4961-a1ff-c26cea04968a admin admin] PATH_INFO: `/v3/projects/f79c5b00a8164dec8db277f492af236d/users/b54a8aed950749b78263986ca93511d4/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:24.118035 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a8b44-2c1c-4961-a1ff-c26cea04968a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f79c5b00a8164dec8db277f492af236d, user_id=b54a8aed950749b78263986ca93511d4, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:24.119451 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8a8b44-2c1c-4961-a1ff-c26cea04968a admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:24.136719 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1398/2782] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:04:24 2022] PUT /identity/v3/projects/f79c5b00a8164dec8db277f492af236d/users/b54a8aed950749b78263986ca93511d4/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:04:24.218388 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed11472-41bc-49bf-a3fc-b313f7be88ec None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:24.218683 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed11472-41bc-49bf-a3fc-b313f7be88ec None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:24.218960 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed11472-41bc-49bf-a3fc-b313f7be88ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:24.260260 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-2ed11472-41bc-49bf-a3fc-b313f7be88ec None None] MFA Rules not processed for user `b54a8aed950749b78263986ca93511d4`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:24.262068 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-2ed11472-41bc-49bf-a3fc-b313f7be88ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:24.346607 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1385/2783] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:04:24 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:04:24.357394 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a8b44-2c1c-4961-a1ff-c26cea04968a admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:24.359210 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-0a8a8b44-2c1c-4961-a1ff-c26cea04968a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:24.402603 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:24.430642 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8a8b44-2c1c-4961-a1ff-c26cea04968a admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:24.434592 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa76512e-eea6-4236-86df-942fb45302f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:24.435331 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76512e-eea6-4236-86df-942fb45302f7 service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:24.435678 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76512e-eea6-4236-86df-942fb45302f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:24.436049 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa76512e-eea6-4236-86df-942fb45302f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:24.452264 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa76512e-eea6-4236-86df-942fb45302f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:24.453774 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa76512e-eea6-4236-86df-942fb45302f7 service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:24.465201 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1399/2784] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:04:24 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:04:25.091205 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed11472-41bc-49bf-a3fc-b313f7be88ec None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:25.180170 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed11472-41bc-49bf-a3fc-b313f7be88ec None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:25.184916 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc9608-ada7-4687-9e54-ffe64428d143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:25.185650 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc9608-ada7-4687-9e54-ffe64428d143 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:25.185931 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc9608-ada7-4687-9e54-ffe64428d143 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:25.186205 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bc9608-ada7-4687-9e54-ffe64428d143 admin admin] PATH_INFO: `/v3/users/1b341bf52b50475bbd6a2d7792bcb92e` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:25.187428 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc9608-ada7-4687-9e54-ffe64428d143 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b341bf52b50475bbd6a2d7792bcb92e)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:25.189363 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bc9608-ada7-4687-9e54-ffe64428d143 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:25.338636 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:25.353122 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-d4bc9608-ada7-4687-9e54-ffe64428d143 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b341bf52b50475bbd6a2d7792bcb92e'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:25.354168 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-d4bc9608-ada7-4687-9e54-ffe64428d143 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b341bf52b50475bbd6a2d7792bcb92e'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:25.420881 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-d4bc9608-ada7-4687-9e54-ffe64428d143 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b341bf52b50475bbd6a2d7792bcb92e'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:25.429726 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-d4bc9608-ada7-4687-9e54-ffe64428d143 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b341bf52b50475bbd6a2d7792bcb92e'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:25.446011 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-d4bc9608-ada7-4687-9e54-ffe64428d143 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b341bf52b50475bbd6a2d7792bcb92e'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:25.461922 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1386/2785] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:04:25 2022] DELETE /identity/v3/users/1b341bf52b50475bbd6a2d7792bcb92e => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:04:25.467407 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d98553-dd28-49f4-a9c0-391dc9295368 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:25.467968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d98553-dd28-49f4-a9c0-391dc9295368 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:25.467968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d98553-dd28-49f4-a9c0-391dc9295368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:25.490026 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-f6d98553-dd28-49f4-a9c0-391dc9295368 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:25.496177 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f6d98553-dd28-49f4-a9c0-391dc9295368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:25.575329 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1400/2786] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:04:25 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:04:25.583758 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc9608-ada7-4687-9e54-ffe64428d143 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:25.585751 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-d4bc9608-ada7-4687-9e54-ffe64428d143 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:25.674223 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bc9608-ada7-4687-9e54-ffe64428d143 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:25.679684 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45c93d-fa76-4134-93a9-341ad20079a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:25.680492 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45c93d-fa76-4134-93a9-341ad20079a4 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:25.681089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45c93d-fa76-4134-93a9-341ad20079a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:25.681459 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d45c93d-fa76-4134-93a9-341ad20079a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:25.697178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d45c93d-fa76-4134-93a9-341ad20079a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:25.699541 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d45c93d-fa76-4134-93a9-341ad20079a4 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:25.717350 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1387/2787] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:04:25 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:04:26.281106 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d98553-dd28-49f4-a9c0-391dc9295368 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:26.292253 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f6d98553-dd28-49f4-a9c0-391dc9295368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:26.299853 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d98553-dd28-49f4-a9c0-391dc9295368 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:26.307961 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d10a9-03ed-4ac4-be46-25ecdcb28828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:26.308968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d10a9-03ed-4ac4-be46-25ecdcb28828 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:26.309449 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d10a9-03ed-4ac4-be46-25ecdcb28828 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:26.310141 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d10a9-03ed-4ac4-be46-25ecdcb28828 admin admin] PATH_INFO: `/v3/projects/29e31e4614f945dea9d528e9e91b673c` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:26.311723 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d10a9-03ed-4ac4-be46-25ecdcb28828 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29e31e4614f945dea9d528e9e91b673c)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:26.314236 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d10a9-03ed-4ac4-be46-25ecdcb28828 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:26.483512 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-bf4d10a9-03ed-4ac4-be46-25ecdcb28828 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29e31e4614f945dea9d528e9e91b673c', 'request_id': 'req-bf4d10a9-03ed-4ac4-be46-25ecdcb28828'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:26.580449 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-bf4d10a9-03ed-4ac4-be46-25ecdcb28828 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29e31e4614f945dea9d528e9e91b673c', 'request_id': 'req-bf4d10a9-03ed-4ac4-be46-25ecdcb28828'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:26.593127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-bf4d10a9-03ed-4ac4-be46-25ecdcb28828 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29e31e4614f945dea9d528e9e91b673c', 'request_id': 'req-bf4d10a9-03ed-4ac4-be46-25ecdcb28828'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:26.614345 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-bf4d10a9-03ed-4ac4-be46-25ecdcb28828 admin admin] The token cache is being invalidate because project 29e31e4614f945dea9d528e9e91b673c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:04:26.614772 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-bf4d10a9-03ed-4ac4-be46-25ecdcb28828 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:26.615682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-bf4d10a9-03ed-4ac4-be46-25ecdcb28828 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:26.617934 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1401/2788] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:04:26 2022] DELETE /identity/v3/projects/29e31e4614f945dea9d528e9e91b673c => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:04:26.621731 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45c93d-fa76-4134-93a9-341ad20079a4 service neutron] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:26.637554 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d45c93d-fa76-4134-93a9-341ad20079a4 service neutron] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:26.741950 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268e72e0-04cb-4801-b41e-9c77ea832ff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:26.744050 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e72e0-04cb-4801-b41e-9c77ea832ff9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:26.751192 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e72e0-04cb-4801-b41e-9c77ea832ff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:26.751192 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268e72e0-04cb-4801-b41e-9c77ea832ff9 admin admin] PATH_INFO: `/v3/users/b54a8aed950749b78263986ca93511d4` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:26.751192 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e72e0-04cb-4801-b41e-9c77ea832ff9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b54a8aed950749b78263986ca93511d4)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:26.752735 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268e72e0-04cb-4801-b41e-9c77ea832ff9 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:27.073945 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-268e72e0-04cb-4801-b41e-9c77ea832ff9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b54a8aed950749b78263986ca93511d4'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:27.075506 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-268e72e0-04cb-4801-b41e-9c77ea832ff9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b54a8aed950749b78263986ca93511d4'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:27.404038 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-268e72e0-04cb-4801-b41e-9c77ea832ff9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b54a8aed950749b78263986ca93511d4'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:27.404038 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:27.406834 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-268e72e0-04cb-4801-b41e-9c77ea832ff9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b54a8aed950749b78263986ca93511d4'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:27.418879 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-268e72e0-04cb-4801-b41e-9c77ea832ff9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b54a8aed950749b78263986ca93511d4'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:27.424892 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1388/2789] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:04:26 2022] DELETE /identity/v3/users/b54a8aed950749b78263986ca93511d4 => generated 0 bytes in 804 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:04:27.917645 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d10a9-03ed-4ac4-be46-25ecdcb28828 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:27.919359 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-bf4d10a9-03ed-4ac4-be46-25ecdcb28828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:27.939445 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d10a9-03ed-4ac4-be46-25ecdcb28828 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:28.009349 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2186183-e821-4007-82e5-0298b9d7b52c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:28.010031 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2186183-e821-4007-82e5-0298b9d7b52c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:28.010311 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2186183-e821-4007-82e5-0298b9d7b52c admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:28.010600 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2186183-e821-4007-82e5-0298b9d7b52c admin admin] PATH_INFO: `/v3/projects/f79c5b00a8164dec8db277f492af236d` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:28.012204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2186183-e821-4007-82e5-0298b9d7b52c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f79c5b00a8164dec8db277f492af236d)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:28.014430 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2186183-e821-4007-82e5-0298b9d7b52c admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:28.016289 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:28.143272 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-b2186183-e821-4007-82e5-0298b9d7b52c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f79c5b00a8164dec8db277f492af236d', 'request_id': 'req-b2186183-e821-4007-82e5-0298b9d7b52c'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:28.206335 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-b2186183-e821-4007-82e5-0298b9d7b52c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f79c5b00a8164dec8db277f492af236d', 'request_id': 'req-b2186183-e821-4007-82e5-0298b9d7b52c'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:28.227414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-b2186183-e821-4007-82e5-0298b9d7b52c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f79c5b00a8164dec8db277f492af236d', 'request_id': 'req-b2186183-e821-4007-82e5-0298b9d7b52c'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:28.253318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-b2186183-e821-4007-82e5-0298b9d7b52c admin admin] The token cache is being invalidate because project f79c5b00a8164dec8db277f492af236d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:04:28.279460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-b2186183-e821-4007-82e5-0298b9d7b52c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:28.279460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-b2186183-e821-4007-82e5-0298b9d7b52c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:28.279460 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1402/2790] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:04:27 2022] DELETE /identity/v3/projects/f79c5b00a8164dec8db277f492af236d => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:04:28.279860 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e11570-3033-4831-8ebe-793992eb9783 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:28.279860 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e11570-3033-4831-8ebe-793992eb9783 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:28.279860 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e11570-3033-4831-8ebe-793992eb9783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:28.283335 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-e9e11570-3033-4831-8ebe-793992eb9783 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:28.286351 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-e9e11570-3033-4831-8ebe-793992eb9783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:28.358118 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1389/2791] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:04:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:04:28.393691 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffcc0de-b304-4022-a282-5cc8622ee7e0 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:28.393974 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffcc0de-b304-4022-a282-5cc8622ee7e0 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:28.394572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffcc0de-b304-4022-a282-5cc8622ee7e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:28.415542 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-4ffcc0de-b304-4022-a282-5cc8622ee7e0 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:28.416749 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-4ffcc0de-b304-4022-a282-5cc8622ee7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:28.424761 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1403/2792] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:04:28 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:04:28.430074 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e11570-3033-4831-8ebe-793992eb9783 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:28.440315 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e11570-3033-4831-8ebe-793992eb9783 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:28.445380 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdfd0f4-20ef-4b5a-981e-bd936281c27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:28.446084 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfd0f4-20ef-4b5a-981e-bd936281c27c admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:28.446326 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfd0f4-20ef-4b5a-981e-bd936281c27c admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:28.447027 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdfd0f4-20ef-4b5a-981e-bd936281c27c admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:28.447337 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdfd0f4-20ef-4b5a-981e-bd936281c27c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:04:28.454140 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdfd0f4-20ef-4b5a-981e-bd936281c27c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:28.454140 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdfd0f4-20ef-4b5a-981e-bd936281c27c admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:28.462765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1390/2793] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 11:04:28 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:04:28.467382 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffcc0de-b304-4022-a282-5cc8622ee7e0 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:28.479587 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffcc0de-b304-4022-a282-5cc8622ee7e0 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:28.487073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917aa26-f4d1-49c8-b348-2e0c37c0a68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:28.487895 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917aa26-f4d1-49c8-b348-2e0c37c0a68b admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:28.488260 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917aa26-f4d1-49c8-b348-2e0c37c0a68b admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:28.488602 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917aa26-f4d1-49c8-b348-2e0c37c0a68b admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:28.489312 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f917aa26-f4d1-49c8-b348-2e0c37c0a68b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:28.491337 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f917aa26-f4d1-49c8-b348-2e0c37c0a68b admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:28.600172 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1404/2794] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:04:28 2022] POST /identity/v3/projects => generated 386 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:04:28.604574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdfd0f4-20ef-4b5a-981e-bd936281c27c admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:28.615302 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdfd0f4-20ef-4b5a-981e-bd936281c27c admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:28.676252 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09658e37-d248-4c5a-bb54-59ecc3f8be80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:28.676870 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09658e37-d248-4c5a-bb54-59ecc3f8be80 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:28.677152 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09658e37-d248-4c5a-bb54-59ecc3f8be80 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:28.677437 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09658e37-d248-4c5a-bb54-59ecc3f8be80 admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:28.678114 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09658e37-d248-4c5a-bb54-59ecc3f8be80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:28.680065 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09658e37-d248-4c5a-bb54-59ecc3f8be80 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:28.779479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1391/2795] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:04:28 2022] POST /identity/v3/users => generated 294 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:04:28.780549 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917aa26-f4d1-49c8-b348-2e0c37c0a68b admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:28.792002 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f917aa26-f4d1-49c8-b348-2e0c37c0a68b admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:28.796897 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a00ed5-5e10-4852-acc2-7753345d33aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:28.797587 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a00ed5-5e10-4852-acc2-7753345d33aa admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:28.797864 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a00ed5-5e10-4852-acc2-7753345d33aa admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:28.798134 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a00ed5-5e10-4852-acc2-7753345d33aa admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:28.798818 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a00ed5-5e10-4852-acc2-7753345d33aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:28.800756 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a00ed5-5e10-4852-acc2-7753345d33aa admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:28.811624 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1405/2796] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:04:28 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:04:28.816595 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09658e37-d248-4c5a-bb54-59ecc3f8be80 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:28.827244 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09658e37-d248-4c5a-bb54-59ecc3f8be80 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:28.833096 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97de198-5538-439f-aee3-00c304a6f2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:28.833978 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97de198-5538-439f-aee3-00c304a6f2e4 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:28.834131 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97de198-5538-439f-aee3-00c304a6f2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:28.835229 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97de198-5538-439f-aee3-00c304a6f2e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:28.835780 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97de198-5538-439f-aee3-00c304a6f2e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:28.838315 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97de198-5538-439f-aee3-00c304a6f2e4 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:28.849692 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1392/2797] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:04:28 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:04:28.853945 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a00ed5-5e10-4852-acc2-7753345d33aa admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:28.864297 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a00ed5-5e10-4852-acc2-7753345d33aa admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:28.868862 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f315324e-2459-495b-9489-4ebddcc40c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:28.869506 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315324e-2459-495b-9489-4ebddcc40c39 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:28.869781 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315324e-2459-495b-9489-4ebddcc40c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:28.870051 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f315324e-2459-495b-9489-4ebddcc40c39 admin admin] PATH_INFO: `/v3/projects/6e2c965bec5a412fa6cc1785cd46d7b9/users/c1afb29c0dff4bb8aad7a9d26dae7e69/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:28.885058 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f315324e-2459-495b-9489-4ebddcc40c39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e2c965bec5a412fa6cc1785cd46d7b9, user_id=c1afb29c0dff4bb8aad7a9d26dae7e69, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:28.887739 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f315324e-2459-495b-9489-4ebddcc40c39 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:28.998007 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1406/2798] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:04:28 2022] PUT /identity/v3/projects/6e2c965bec5a412fa6cc1785cd46d7b9/users/c1afb29c0dff4bb8aad7a9d26dae7e69/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:04:29.018098 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaea209-e2a4-4be2-9431-0b92e63568cf None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:29.018380 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaea209-e2a4-4be2-9431-0b92e63568cf None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:29.020230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aaea209-e2a4-4be2-9431-0b92e63568cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:29.054322 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-4aaea209-e2a4-4be2-9431-0b92e63568cf None None] MFA Rules not processed for user `c1afb29c0dff4bb8aad7a9d26dae7e69`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:29.055635 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-4aaea209-e2a4-4be2-9431-0b92e63568cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:29.098647 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:29.138106 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1393/2799] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:04:29 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:04:29.142654 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f315324e-2459-495b-9489-4ebddcc40c39 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:29.208704 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f315324e-2459-495b-9489-4ebddcc40c39 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:29.213503 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0433e4b5-9a03-488b-b170-2d53ced8d046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:29.214169 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433e4b5-9a03-488b-b170-2d53ced8d046 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:29.214453 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433e4b5-9a03-488b-b170-2d53ced8d046 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:29.214729 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0433e4b5-9a03-488b-b170-2d53ced8d046 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:29.215430 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0433e4b5-9a03-488b-b170-2d53ced8d046 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:29.217366 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0433e4b5-9a03-488b-b170-2d53ced8d046 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:29.303698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1407/2800] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:04:29 2022] POST /identity/v3/projects => generated 384 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:04:29.308116 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaea209-e2a4-4be2-9431-0b92e63568cf None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:29.320243 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aaea209-e2a4-4be2-9431-0b92e63568cf None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:29.379853 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607cce3a-8df6-4481-81cf-dcf3a759ffab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:29.380479 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607cce3a-8df6-4481-81cf-dcf3a759ffab admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:29.380762 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607cce3a-8df6-4481-81cf-dcf3a759ffab admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:29.381039 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607cce3a-8df6-4481-81cf-dcf3a759ffab admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:29.381718 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607cce3a-8df6-4481-81cf-dcf3a759ffab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:29.383922 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607cce3a-8df6-4481-81cf-dcf3a759ffab admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:29.512346 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1394/2801] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:04:29 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:04:29.516590 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0433e4b5-9a03-488b-b170-2d53ced8d046 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:29.527894 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0433e4b5-9a03-488b-b170-2d53ced8d046 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:29.532495 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431cc44-d3f0-44a0-ae6c-2984aaacd78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:29.533092 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431cc44-d3f0-44a0-ae6c-2984aaacd78e admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:29.537245 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431cc44-d3f0-44a0-ae6c-2984aaacd78e admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:29.537245 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431cc44-d3f0-44a0-ae6c-2984aaacd78e admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:29.537245 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7431cc44-d3f0-44a0-ae6c-2984aaacd78e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:29.537245 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7431cc44-d3f0-44a0-ae6c-2984aaacd78e admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:29.542036 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:29.547601 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1408/2802] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:04:29 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:04:29.552049 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607cce3a-8df6-4481-81cf-dcf3a759ffab admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:29.566115 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607cce3a-8df6-4481-81cf-dcf3a759ffab admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:29.568812 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d22b78a-5017-4f5f-a0f9-234a447ebbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:29.569432 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22b78a-5017-4f5f-a0f9-234a447ebbcd admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:29.569713 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22b78a-5017-4f5f-a0f9-234a447ebbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:29.569986 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d22b78a-5017-4f5f-a0f9-234a447ebbcd admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:29.570680 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d22b78a-5017-4f5f-a0f9-234a447ebbcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:29.572301 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d22b78a-5017-4f5f-a0f9-234a447ebbcd admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:29.583359 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1395/2803] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:04:29 2022] GET /identity/v3/roles => generated 1427 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:04:29.588156 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431cc44-d3f0-44a0-ae6c-2984aaacd78e admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:29.599924 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431cc44-d3f0-44a0-ae6c-2984aaacd78e admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:29.604604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f0ee9e-4c33-48a4-9e88-9aa851dcf703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:29.605418 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0ee9e-4c33-48a4-9e88-9aa851dcf703 admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:29.605525 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0ee9e-4c33-48a4-9e88-9aa851dcf703 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:29.605840 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f0ee9e-4c33-48a4-9e88-9aa851dcf703 admin admin] PATH_INFO: `/v3/projects/a4e7a48602b44e33b84855f19be1cce2/users/6d59154a9d9648ad8a04c0c637c2cb9b/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:29.623056 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0ee9e-4c33-48a4-9e88-9aa851dcf703 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4e7a48602b44e33b84855f19be1cce2, user_id=6d59154a9d9648ad8a04c0c637c2cb9b, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:29.625972 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f0ee9e-4c33-48a4-9e88-9aa851dcf703 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:29.729648 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1409/2804] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:04:29 2022] PUT /identity/v3/projects/a4e7a48602b44e33b84855f19be1cce2/users/6d59154a9d9648ad8a04c0c637c2cb9b/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:04:29.740702 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87ce98-91f5-45fa-b5ff-4c79a2ee1ee1 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:29.740989 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87ce98-91f5-45fa-b5ff-4c79a2ee1ee1 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:29.741272 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a87ce98-91f5-45fa-b5ff-4c79a2ee1ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:29.775544 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-9a87ce98-91f5-45fa-b5ff-4c79a2ee1ee1 None None] MFA Rules not processed for user `6d59154a9d9648ad8a04c0c637c2cb9b`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:29.775817 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da452e25-c28b-481d-bd0d-ab4b5a50c86c None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:29.776109 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da452e25-c28b-481d-bd0d-ab4b5a50c86c None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:29.776389 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da452e25-c28b-481d-bd0d-ab4b5a50c86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:29.776745 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-9a87ce98-91f5-45fa-b5ff-4c79a2ee1ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:29.796682 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-da452e25-c28b-481d-bd0d-ab4b5a50c86c None None] MFA Rules not processed for user `2b03263937cb4ca7807a24da44d37b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:29.797895 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-da452e25-c28b-481d-bd0d-ab4b5a50c86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:29.843808 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1396/2805] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:04:29 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:04:29.846642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87ce98-91f5-45fa-b5ff-4c79a2ee1ee1 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:29.854402 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1410/2806] 10.222.0.115 () {58 vars in 1014 bytes} [Tue Aug 2 11:04:29 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:04:29.866655 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da452e25-c28b-481d-bd0d-ab4b5a50c86c None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:29.868050 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-da452e25-c28b-481d-bd0d-ab4b5a50c86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:29.891772 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:29.917059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a87ce98-91f5-45fa-b5ff-4c79a2ee1ee1 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:29.921721 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de696d31-5573-4aeb-b517-cd843e3c7836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:29.922341 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de696d31-5573-4aeb-b517-cd843e3c7836 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:29.922620 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de696d31-5573-4aeb-b517-cd843e3c7836 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:29.922897 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de696d31-5573-4aeb-b517-cd843e3c7836 admin admin] PATH_INFO: `/v3/projects` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:29.923594 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de696d31-5573-4aeb-b517-cd843e3c7836 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:29.926324 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de696d31-5573-4aeb-b517-cd843e3c7836 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:29.941002 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da452e25-c28b-481d-bd0d-ab4b5a50c86c None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:29.946223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2160908-a1e7-4e46-871c-473d4872c07f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0daa49b5a35f4a468bb8d0939e7ec20c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:29.946841 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2160908-a1e7-4e46-871c-473d4872c07f service swift] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:29.947163 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2160908-a1e7-4e46-871c-473d4872c07f service swift] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:29.947442 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2160908-a1e7-4e46-871c-473d4872c07f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:29.956809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2160908-a1e7-4e46-871c-473d4872c07f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:29.956809 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2160908-a1e7-4e46-871c-473d4872c07f service swift] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:29.967232 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1411/2807] 10.222.0.115 () {60 vars in 1389 bytes} [Tue Aug 2 11:04:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 02 11:04:30.044160 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1397/2808] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:04:29 2022] POST /identity/v3/projects => generated 386 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:04:30.048010 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2160908-a1e7-4e46-871c-473d4872c07f service swift] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:30.058742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2160908-a1e7-4e46-871c-473d4872c07f service swift] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:30.136408 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a85f85-8907-4e2c-bc8a-3a6fd1296325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:30.137015 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a85f85-8907-4e2c-bc8a-3a6fd1296325 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:30.137293 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a85f85-8907-4e2c-bc8a-3a6fd1296325 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:30.137562 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a85f85-8907-4e2c-bc8a-3a6fd1296325 admin admin] PATH_INFO: `/v3/users` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:30.138265 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a85f85-8907-4e2c-bc8a-3a6fd1296325 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:30.140203 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a85f85-8907-4e2c-bc8a-3a6fd1296325 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:30.189039 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ec9bf-0da2-4523-9063-a6d7e01724f7 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:30.189328 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ec9bf-0da2-4523-9063-a6d7e01724f7 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:30.189608 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36ec9bf-0da2-4523-9063-a6d7e01724f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:30.212132 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-f36ec9bf-0da2-4523-9063-a6d7e01724f7 None None] MFA Rules not processed for user `2b03263937cb4ca7807a24da44d37b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:30.213420 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-f36ec9bf-0da2-4523-9063-a6d7e01724f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:30.248294 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:30.264107 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1412/2809] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:04:30 2022] POST /identity/v3/users => generated 300 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:04:30.265338 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a85f85-8907-4e2c-bc8a-3a6fd1296325 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:30.276935 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a85f85-8907-4e2c-bc8a-3a6fd1296325 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:30.282073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766badd-50dd-4cbb-a76f-67076f6a93e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:30.285188 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766badd-50dd-4cbb-a76f-67076f6a93e0 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:30.285188 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766badd-50dd-4cbb-a76f-67076f6a93e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:30.285188 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7766badd-50dd-4cbb-a76f-67076f6a93e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:30.285188 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7766badd-50dd-4cbb-a76f-67076f6a93e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:30.285829 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1398/2810] 10.222.0.115 () {58 vars in 972 bytes} [Tue Aug 2 11:04:30 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:04:30.286819 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7766badd-50dd-4cbb-a76f-67076f6a93e0 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:30.298159 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1413/2811] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:04:30 2022] GET /identity/v3/roles => generated 1427 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:04:30.298868 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ec9bf-0da2-4523-9063-a6d7e01724f7 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:30.302294 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766badd-50dd-4cbb-a76f-67076f6a93e0 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:30.312741 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7766badd-50dd-4cbb-a76f-67076f6a93e0 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:30.318097 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a827afe-283e-43db-893f-4fe446686dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:30.318833 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a827afe-283e-43db-893f-4fe446686dca admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:30.319100 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a827afe-283e-43db-893f-4fe446686dca admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:30.319380 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a827afe-283e-43db-893f-4fe446686dca admin admin] PATH_INFO: `/v3/projects/bb47c2435df64f84bc22c7b0578ce711/users/edbe62909b0e42928e27a32e28ffc67a/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:30.336788 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a827afe-283e-43db-893f-4fe446686dca admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb47c2435df64f84bc22c7b0578ce711, user_id=edbe62909b0e42928e27a32e28ffc67a, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:30.339738 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a827afe-283e-43db-893f-4fe446686dca admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:30.369551 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36ec9bf-0da2-4523-9063-a6d7e01724f7 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:30.375133 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a73f79-ee0b-42a0-aa4f-18941f96b117 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0daa49b5a35f4a468bb8d0939e7ec20c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:30.375766 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a73f79-ee0b-42a0-aa4f-18941f96b117 service swift] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:30.376049 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a73f79-ee0b-42a0-aa4f-18941f96b117 service swift] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:30.376326 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a73f79-ee0b-42a0-aa4f-18941f96b117 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:30.386030 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a73f79-ee0b-42a0-aa4f-18941f96b117 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:30.387724 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a73f79-ee0b-42a0-aa4f-18941f96b117 service swift] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:30.397051 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1399/2812] 10.222.0.115 () {60 vars in 1389 bytes} [Tue Aug 2 11:04:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 02 11:04:30.433925 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1414/2813] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:04:30 2022] PUT /identity/v3/projects/bb47c2435df64f84bc22c7b0578ce711/users/edbe62909b0e42928e27a32e28ffc67a/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:04:30.437626 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a73f79-ee0b-42a0-aa4f-18941f96b117 service swift] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:30.448968 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a73f79-ee0b-42a0-aa4f-18941f96b117 service swift] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:30.504120 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:30.511936 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1df351-68ba-4770-a28b-5c7bf3cf6377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:30.512643 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1df351-68ba-4770-a28b-5c7bf3cf6377 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:30.513232 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1df351-68ba-4770-a28b-5c7bf3cf6377 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:30.513232 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1df351-68ba-4770-a28b-5c7bf3cf6377 admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:30.513870 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1df351-68ba-4770-a28b-5c7bf3cf6377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:30.515498 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1df351-68ba-4770-a28b-5c7bf3cf6377 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:30.529267 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1400/2814] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:04:30 2022] GET /identity/v3/roles => generated 1427 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:04:30.532824 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a827afe-283e-43db-893f-4fe446686dca admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:30.544316 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a827afe-283e-43db-893f-4fe446686dca admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:30.549587 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9186a6-e637-43a9-9a23-17c8086bc88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:30.550332 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9186a6-e637-43a9-9a23-17c8086bc88e admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:30.550664 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9186a6-e637-43a9-9a23-17c8086bc88e admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:30.551101 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9186a6-e637-43a9-9a23-17c8086bc88e admin admin] PATH_INFO: `/v3/projects/bb47c2435df64f84bc22c7b0578ce711/users/edbe62909b0e42928e27a32e28ffc67a/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:30.553295 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9186a6-e637-43a9-9a23-17c8086bc88e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb47c2435df64f84bc22c7b0578ce711, user_id=edbe62909b0e42928e27a32e28ffc67a, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:30.556028 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9186a6-e637-43a9-9a23-17c8086bc88e admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:30.569151 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1415/2815] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:04:30 2022] PUT /identity/v3/projects/bb47c2435df64f84bc22c7b0578ce711/users/edbe62909b0e42928e27a32e28ffc67a/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:04:30.579313 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307212fe-8fa4-4f0b-9d64-9f021d2977bf None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:30.579609 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307212fe-8fa4-4f0b-9d64-9f021d2977bf None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:30.579884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307212fe-8fa4-4f0b-9d64-9f021d2977bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:30.610235 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-307212fe-8fa4-4f0b-9d64-9f021d2977bf None None] MFA Rules not processed for user `edbe62909b0e42928e27a32e28ffc67a`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:30.611524 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-307212fe-8fa4-4f0b-9d64-9f021d2977bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:30.703581 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1401/2816] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:04:30 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:04:30.716433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9186a6-e637-43a9-9a23-17c8086bc88e admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:30.718230 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-fd9186a6-e637-43a9-9a23-17c8086bc88e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:30.785468 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9186a6-e637-43a9-9a23-17c8086bc88e admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:30.791204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28a30f-0b1c-46d1-9d0b-d5f6ede65d59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:30.791204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28a30f-0b1c-46d1-9d0b-d5f6ede65d59 service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:30.791204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28a30f-0b1c-46d1-9d0b-d5f6ede65d59 service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:30.791204 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb28a30f-0b1c-46d1-9d0b-d5f6ede65d59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:30.861605 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb28a30f-0b1c-46d1-9d0b-d5f6ede65d59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:30.863743 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb28a30f-0b1c-46d1-9d0b-d5f6ede65d59 service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:30.873584 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1416/2817] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:04:30 2022] GET /identity/v3/auth/tokens => generated 3272 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:04:31.021900 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb7119-dbb1-46e7-9537-aca9d88cfce8 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:31.022194 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb7119-dbb1-46e7-9537-aca9d88cfce8 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:31.022635 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb7119-dbb1-46e7-9537-aca9d88cfce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:31.043317 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-1fcb7119-dbb1-46e7-9537-aca9d88cfce8 None None] MFA Rules not processed for user `2b03263937cb4ca7807a24da44d37b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:31.044584 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-1fcb7119-dbb1-46e7-9537-aca9d88cfce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:31.101998 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1402/2818] 10.222.0.115 () {58 vars in 972 bytes} [Tue Aug 2 11:04:31 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:04:31.114358 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28a30f-0b1c-46d1-9d0b-d5f6ede65d59 service cinder] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:31.182346 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb28a30f-0b1c-46d1-9d0b-d5f6ede65d59 service cinder] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:31.188623 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb551d4-49d8-4063-a0eb-4325be36caf4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0daa49b5a35f4a468bb8d0939e7ec20c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:31.189241 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb551d4-49d8-4063-a0eb-4325be36caf4 service swift] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:31.189519 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb551d4-49d8-4063-a0eb-4325be36caf4 service swift] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:31.189788 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb551d4-49d8-4063-a0eb-4325be36caf4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:31.200569 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:31.203127 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb551d4-49d8-4063-a0eb-4325be36caf4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:31.205134 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb551d4-49d8-4063-a0eb-4325be36caf4 service swift] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:31.214589 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1417/2819] 10.222.0.115 () {60 vars in 1389 bytes} [Tue Aug 2 11:04:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 11:04:31.874394 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f130e5-570b-4fd9-b35f-32db0e5e1301 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:31.874865 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f130e5-570b-4fd9-b35f-32db0e5e1301 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:31.874980 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f130e5-570b-4fd9-b35f-32db0e5e1301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:31.894771 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-d5f130e5-570b-4fd9-b35f-32db0e5e1301 None None] MFA Rules not processed for user `2b03263937cb4ca7807a24da44d37b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:31.896011 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-d5f130e5-570b-4fd9-b35f-32db0e5e1301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:31.903150 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1403/2820] 10.222.0.115 () {58 vars in 1014 bytes} [Tue Aug 2 11:04:31 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:04:31.915672 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb551d4-49d8-4063-a0eb-4325be36caf4 service swift] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:31.926432 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb551d4-49d8-4063-a0eb-4325be36caf4 service swift] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:31.931413 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f71cf-ec10-485f-8a53-29225a26092a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0daa49b5a35f4a468bb8d0939e7ec20c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:31.932017 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f71cf-ec10-485f-8a53-29225a26092a service swift] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:31.932316 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f71cf-ec10-485f-8a53-29225a26092a service swift] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:31.932585 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67f71cf-ec10-485f-8a53-29225a26092a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:31.941530 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f71cf-ec10-485f-8a53-29225a26092a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:31.943201 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67f71cf-ec10-485f-8a53-29225a26092a service swift] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:31.951461 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1418/2821] 10.222.0.115 () {60 vars in 1389 bytes} [Tue Aug 2 11:04:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 11:04:32.539926 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa23aaef-7bc4-45ac-8dc9-d292510aa503 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:32.540218 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa23aaef-7bc4-45ac-8dc9-d292510aa503 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:32.540506 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa23aaef-7bc4-45ac-8dc9-d292510aa503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:32.560208 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-fa23aaef-7bc4-45ac-8dc9-d292510aa503 None None] MFA Rules not processed for user `2b03263937cb4ca7807a24da44d37b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:32.561833 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-fa23aaef-7bc4-45ac-8dc9-d292510aa503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:32.570121 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1404/2822] 10.222.0.115 () {58 vars in 972 bytes} [Tue Aug 2 11:04:32 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:04:32.583601 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f71cf-ec10-485f-8a53-29225a26092a service swift] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:32.595274 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67f71cf-ec10-485f-8a53-29225a26092a service swift] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:32.600469 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5823ba64-fdcd-4bd1-9db9-ff91c85e174c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0daa49b5a35f4a468bb8d0939e7ec20c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:32.601072 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823ba64-fdcd-4bd1-9db9-ff91c85e174c service swift] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:32.601375 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823ba64-fdcd-4bd1-9db9-ff91c85e174c service swift] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:32.601643 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5823ba64-fdcd-4bd1-9db9-ff91c85e174c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:32.610107 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5823ba64-fdcd-4bd1-9db9-ff91c85e174c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:32.611789 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5823ba64-fdcd-4bd1-9db9-ff91c85e174c service swift] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:32.620552 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1419/2823] 10.222.0.115 () {60 vars in 1389 bytes} [Tue Aug 2 11:04:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 11:04:32.922464 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa23aaef-7bc4-45ac-8dc9-d292510aa503 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:32.932917 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa23aaef-7bc4-45ac-8dc9-d292510aa503 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:32.938946 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa42449e-afe8-4b68-a529-c8b9c6de24f9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:32.938946 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42449e-afe8-4b68-a529-c8b9c6de24f9 service cinder] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:32.938946 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42449e-afe8-4b68-a529-c8b9c6de24f9 service cinder] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:32.938946 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa42449e-afe8-4b68-a529-c8b9c6de24f9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:33.007297 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa42449e-afe8-4b68-a529-c8b9c6de24f9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:33.007297 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa42449e-afe8-4b68-a529-c8b9c6de24f9 service cinder] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:33.016172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1405/2824] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:04:32 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:04:33.486637 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5823ba64-fdcd-4bd1-9db9-ff91c85e174c service swift] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:33.489160 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-5823ba64-fdcd-4bd1-9db9-ff91c85e174c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:33.595344 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5823ba64-fdcd-4bd1-9db9-ff91c85e174c service swift] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:33.601506 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46fe49-3e7f-4603-8e64-ce2c161f1ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:33.602496 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46fe49-3e7f-4603-8e64-ce2c161f1ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:33.602855 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46fe49-3e7f-4603-8e64-ce2c161f1ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:33.603386 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d46fe49-3e7f-4603-8e64-ce2c161f1ff6 admin admin] PATH_INFO: `/v3/users/d4ba1b5dac82438391a56b7cda25a363` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:33.605441 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d46fe49-3e7f-4603-8e64-ce2c161f1ff6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4ba1b5dac82438391a56b7cda25a363)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:33.607090 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d46fe49-3e7f-4603-8e64-ce2c161f1ff6 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:33.803311 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-4d46fe49-3e7f-4603-8e64-ce2c161f1ff6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4ba1b5dac82438391a56b7cda25a363'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:33.809949 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-4d46fe49-3e7f-4603-8e64-ce2c161f1ff6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4ba1b5dac82438391a56b7cda25a363'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:33.810662 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-4d46fe49-3e7f-4603-8e64-ce2c161f1ff6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4ba1b5dac82438391a56b7cda25a363'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:33.905907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-4d46fe49-3e7f-4603-8e64-ce2c161f1ff6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4ba1b5dac82438391a56b7cda25a363'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:33.925488 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-4d46fe49-3e7f-4603-8e64-ce2c161f1ff6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4ba1b5dac82438391a56b7cda25a363'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:33.931397 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1420/2825] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:04:33 2022] DELETE /identity/v3/users/d4ba1b5dac82438391a56b7cda25a363 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:04:34.197768 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa42449e-afe8-4b68-a529-c8b9c6de24f9 service cinder] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:34.210151 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa42449e-afe8-4b68-a529-c8b9c6de24f9 service cinder] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:34.213970 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=100 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:34.278307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e22e4-3daa-4336-ba67-3e7ae4b97169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:34.278987 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e22e4-3daa-4336-ba67-3e7ae4b97169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:34.279298 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e22e4-3daa-4336-ba67-3e7ae4b97169 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:34.279575 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5e22e4-3daa-4336-ba67-3e7ae4b97169 admin admin] PATH_INFO: `/v3/projects/4dccda1fb9a44c3d91811677bd6f36c3` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:34.280855 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e22e4-3daa-4336-ba67-3e7ae4b97169 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4dccda1fb9a44c3d91811677bd6f36c3)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:34.282758 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5e22e4-3daa-4336-ba67-3e7ae4b97169 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:34.416562 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-aa5e22e4-3daa-4336-ba67-3e7ae4b97169 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4dccda1fb9a44c3d91811677bd6f36c3', 'request_id': 'req-aa5e22e4-3daa-4336-ba67-3e7ae4b97169'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:34.421241 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-aa5e22e4-3daa-4336-ba67-3e7ae4b97169 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4dccda1fb9a44c3d91811677bd6f36c3', 'request_id': 'req-aa5e22e4-3daa-4336-ba67-3e7ae4b97169'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:34.511849 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-aa5e22e4-3daa-4336-ba67-3e7ae4b97169 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4dccda1fb9a44c3d91811677bd6f36c3', 'request_id': 'req-aa5e22e4-3daa-4336-ba67-3e7ae4b97169'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:34.526129 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-aa5e22e4-3daa-4336-ba67-3e7ae4b97169 admin admin] The token cache is being invalidate because project 4dccda1fb9a44c3d91811677bd6f36c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:04:34.526480 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-aa5e22e4-3daa-4336-ba67-3e7ae4b97169 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:34.527089 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-aa5e22e4-3daa-4336-ba67-3e7ae4b97169 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:34.534730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1406/2826] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:04:34 2022] DELETE /identity/v3/projects/4dccda1fb9a44c3d91811677bd6f36c3 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:04:34.720644 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46fe49-3e7f-4603-8e64-ce2c161f1ff6 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:34.723160 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-4d46fe49-3e7f-4603-8e64-ce2c161f1ff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:34.789785 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:34.793461 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d46fe49-3e7f-4603-8e64-ce2c161f1ff6 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:34.844359 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea74ead-3a46-4c90-a6fe-76a8a3b8e899 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:34.844981 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea74ead-3a46-4c90-a6fe-76a8a3b8e899 service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:34.845260 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea74ead-3a46-4c90-a6fe-76a8a3b8e899 service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:34.845533 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea74ead-3a46-4c90-a6fe-76a8a3b8e899 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:34.847363 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-cea74ead-3a46-4c90-a6fe-76a8a3b8e899 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:34.910685 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea74ead-3a46-4c90-a6fe-76a8a3b8e899 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:34.912143 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea74ead-3a46-4c90-a6fe-76a8a3b8e899 service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:34.921476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1421/2827] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:04:34 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:04:35.451729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e22e4-3daa-4336-ba67-3e7ae4b97169 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:35.453681 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-aa5e22e4-3daa-4336-ba67-3e7ae4b97169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:35.524652 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5e22e4-3daa-4336-ba67-3e7ae4b97169 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:35.529477 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93427274-e59c-4329-abe8-3f725a2877f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:35.530104 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93427274-e59c-4329-abe8-3f725a2877f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:35.530383 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93427274-e59c-4329-abe8-3f725a2877f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:35.530655 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93427274-e59c-4329-abe8-3f725a2877f6 admin admin] PATH_INFO: `/v3/users/9ab9773545044cbf8e6f0318fe649f9d` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:35.531909 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93427274-e59c-4329-abe8-3f725a2877f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ab9773545044cbf8e6f0318fe649f9d)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:35.533954 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93427274-e59c-4329-abe8-3f725a2877f6 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:35.723022 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:35.729372 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-93427274-e59c-4329-abe8-3f725a2877f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ab9773545044cbf8e6f0318fe649f9d'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:35.730167 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-93427274-e59c-4329-abe8-3f725a2877f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ab9773545044cbf8e6f0318fe649f9d'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:35.797571 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-93427274-e59c-4329-abe8-3f725a2877f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ab9773545044cbf8e6f0318fe649f9d'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:35.806190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-93427274-e59c-4329-abe8-3f725a2877f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ab9773545044cbf8e6f0318fe649f9d'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:35.820440 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-93427274-e59c-4329-abe8-3f725a2877f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ab9773545044cbf8e6f0318fe649f9d'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:35.826477 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1407/2828] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:04:35 2022] DELETE /identity/v3/users/9ab9773545044cbf8e6f0318fe649f9d => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:04:35.831242 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029a80e-a295-45bd-b44b-b9fb93ac76ed None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:35.831783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029a80e-a295-45bd-b44b-b9fb93ac76ed None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:35.831783 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b029a80e-a295-45bd-b44b-b9fb93ac76ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:35.851554 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-b029a80e-a295-45bd-b44b-b9fb93ac76ed None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:35.852762 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-b029a80e-a295-45bd-b44b-b9fb93ac76ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:35.919544 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1422/2829] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:04:35 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:04:35.930538 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93427274-e59c-4329-abe8-3f725a2877f6 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:35.932012 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-93427274-e59c-4329-abe8-3f725a2877f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:35.990722 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93427274-e59c-4329-abe8-3f725a2877f6 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:35.994307 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da6083-8f1e-451f-93f2-c433bf791e12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:35.994925 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da6083-8f1e-451f-93f2-c433bf791e12 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:35.995245 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da6083-8f1e-451f-93f2-c433bf791e12 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:35.995723 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da6083-8f1e-451f-93f2-c433bf791e12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:36.005281 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da6083-8f1e-451f-93f2-c433bf791e12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:36.006912 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da6083-8f1e-451f-93f2-c433bf791e12 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:36.017777 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1408/2830] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:04:35 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:04:36.326102 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b029a80e-a295-45bd-b44b-b9fb93ac76ed None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:36.327634 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-b029a80e-a295-45bd-b44b-b9fb93ac76ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:36.341232 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b029a80e-a295-45bd-b44b-b9fb93ac76ed None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:36.346506 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66051d28-5a5c-406f-a74a-fb9b70f54614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:36.347822 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66051d28-5a5c-406f-a74a-fb9b70f54614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:36.348097 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66051d28-5a5c-406f-a74a-fb9b70f54614 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:36.348531 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66051d28-5a5c-406f-a74a-fb9b70f54614 admin admin] PATH_INFO: `/v3/projects/18a92e65eb7546048be4b7c215257f70` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:36.349633 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66051d28-5a5c-406f-a74a-fb9b70f54614 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18a92e65eb7546048be4b7c215257f70)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:36.351632 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66051d28-5a5c-406f-a74a-fb9b70f54614 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:36.568467 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-66051d28-5a5c-406f-a74a-fb9b70f54614 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18a92e65eb7546048be4b7c215257f70', 'request_id': 'req-66051d28-5a5c-406f-a74a-fb9b70f54614'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:36.570120 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:36.671848 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-66051d28-5a5c-406f-a74a-fb9b70f54614 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18a92e65eb7546048be4b7c215257f70', 'request_id': 'req-66051d28-5a5c-406f-a74a-fb9b70f54614'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:36.678596 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-66051d28-5a5c-406f-a74a-fb9b70f54614 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18a92e65eb7546048be4b7c215257f70', 'request_id': 'req-66051d28-5a5c-406f-a74a-fb9b70f54614'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:36.690681 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-66051d28-5a5c-406f-a74a-fb9b70f54614 admin admin] The token cache is being invalidate because project 18a92e65eb7546048be4b7c215257f70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:04:36.691006 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-66051d28-5a5c-406f-a74a-fb9b70f54614 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:36.691717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-66051d28-5a5c-406f-a74a-fb9b70f54614 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:36.693282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1423/2831] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:04:36 2022] DELETE /identity/v3/projects/18a92e65eb7546048be4b7c215257f70 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:04:45.202475 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da6083-8f1e-451f-93f2-c433bf791e12 service neutron] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:45.203960 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-26da6083-8f1e-451f-93f2-c433bf791e12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:45.216685 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da6083-8f1e-451f-93f2-c433bf791e12 service neutron] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:45.273945 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279676c1-64f6-482b-9c21-82aef4da00e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:45.274575 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279676c1-64f6-482b-9c21-82aef4da00e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:45.274856 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279676c1-64f6-482b-9c21-82aef4da00e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:45.275167 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279676c1-64f6-482b-9c21-82aef4da00e7 admin admin] PATH_INFO: `/v3/users/c1afb29c0dff4bb8aad7a9d26dae7e69` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:45.276496 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279676c1-64f6-482b-9c21-82aef4da00e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1afb29c0dff4bb8aad7a9d26dae7e69)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:45.278540 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279676c1-64f6-482b-9c21-82aef4da00e7 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:45.489548 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-279676c1-64f6-482b-9c21-82aef4da00e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1afb29c0dff4bb8aad7a9d26dae7e69'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:45.490200 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-279676c1-64f6-482b-9c21-82aef4da00e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1afb29c0dff4bb8aad7a9d26dae7e69'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:45.551526 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-279676c1-64f6-482b-9c21-82aef4da00e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1afb29c0dff4bb8aad7a9d26dae7e69'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:45.557181 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:45.559262 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-279676c1-64f6-482b-9c21-82aef4da00e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1afb29c0dff4bb8aad7a9d26dae7e69'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:45.571230 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-279676c1-64f6-482b-9c21-82aef4da00e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1afb29c0dff4bb8aad7a9d26dae7e69'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:45.577157 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1409/2832] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:04:45 2022] DELETE /identity/v3/users/c1afb29c0dff4bb8aad7a9d26dae7e69 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:04:45.581629 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e64d1-5d92-4c03-b3d5-86df15fd2e31 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:45.582119 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e64d1-5d92-4c03-b3d5-86df15fd2e31 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:45.582231 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57e64d1-5d92-4c03-b3d5-86df15fd2e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:45.601348 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-c57e64d1-5d92-4c03-b3d5-86df15fd2e31 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:45.602532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-c57e64d1-5d92-4c03-b3d5-86df15fd2e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:45.662318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1424/2833] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:04:45 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:04:45.672682 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279676c1-64f6-482b-9c21-82aef4da00e7 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:45.675366 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-279676c1-64f6-482b-9c21-82aef4da00e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:45.732900 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279676c1-64f6-482b-9c21-82aef4da00e7 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:45.736599 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7ec65-c725-403e-a759-989470ca865d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:45.738141 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7ec65-c725-403e-a759-989470ca865d service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:45.738141 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7ec65-c725-403e-a759-989470ca865d service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:45.738362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7ec65-c725-403e-a759-989470ca865d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:45.747153 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed7ec65-c725-403e-a759-989470ca865d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:45.748561 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed7ec65-c725-403e-a759-989470ca865d service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:45.758022 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1410/2834] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:04:45 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:04:46.160549 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e64d1-5d92-4c03-b3d5-86df15fd2e31 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:46.160968 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-c57e64d1-5d92-4c03-b3d5-86df15fd2e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:46.174871 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57e64d1-5d92-4c03-b3d5-86df15fd2e31 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:46.179888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534722dc-620f-433f-89df-015526368f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:46.180528 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534722dc-620f-433f-89df-015526368f8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:46.180803 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534722dc-620f-433f-89df-015526368f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:46.181073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534722dc-620f-433f-89df-015526368f8a admin admin] PATH_INFO: `/v3/projects/6e2c965bec5a412fa6cc1785cd46d7b9` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:46.182241 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534722dc-620f-433f-89df-015526368f8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e2c965bec5a412fa6cc1785cd46d7b9)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:46.185083 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534722dc-620f-433f-89df-015526368f8a admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:46.310341 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-534722dc-620f-433f-89df-015526368f8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e2c965bec5a412fa6cc1785cd46d7b9', 'request_id': 'req-534722dc-620f-433f-89df-015526368f8a'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:46.406081 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-534722dc-620f-433f-89df-015526368f8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e2c965bec5a412fa6cc1785cd46d7b9', 'request_id': 'req-534722dc-620f-433f-89df-015526368f8a'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:46.412817 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-534722dc-620f-433f-89df-015526368f8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e2c965bec5a412fa6cc1785cd46d7b9', 'request_id': 'req-534722dc-620f-433f-89df-015526368f8a'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:46.423913 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-534722dc-620f-433f-89df-015526368f8a admin admin] The token cache is being invalidate because project 6e2c965bec5a412fa6cc1785cd46d7b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:04:46.424875 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-534722dc-620f-433f-89df-015526368f8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:46.425486 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-534722dc-620f-433f-89df-015526368f8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:46.426989 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1425/2835] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:04:46 2022] DELETE /identity/v3/projects/6e2c965bec5a412fa6cc1785cd46d7b9 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:04:46.430959 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7ec65-c725-403e-a759-989470ca865d service neutron] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:46.441974 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7ec65-c725-403e-a759-989470ca865d service neutron] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:46.498068 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f467b-f07a-46ed-b0a8-e24444be5c35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:46.498704 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f467b-f07a-46ed-b0a8-e24444be5c35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:46.499011 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f467b-f07a-46ed-b0a8-e24444be5c35 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:46.499317 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305f467b-f07a-46ed-b0a8-e24444be5c35 admin admin] PATH_INFO: `/v3/users/6d59154a9d9648ad8a04c0c637c2cb9b` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:46.500474 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f467b-f07a-46ed-b0a8-e24444be5c35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d59154a9d9648ad8a04c0c637c2cb9b)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:46.503190 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305f467b-f07a-46ed-b0a8-e24444be5c35 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:46.706075 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-305f467b-f07a-46ed-b0a8-e24444be5c35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d59154a9d9648ad8a04c0c637c2cb9b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:46.706760 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-305f467b-f07a-46ed-b0a8-e24444be5c35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d59154a9d9648ad8a04c0c637c2cb9b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:46.789087 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-305f467b-f07a-46ed-b0a8-e24444be5c35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d59154a9d9648ad8a04c0c637c2cb9b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:46.796468 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-305f467b-f07a-46ed-b0a8-e24444be5c35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d59154a9d9648ad8a04c0c637c2cb9b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:46.808628 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-305f467b-f07a-46ed-b0a8-e24444be5c35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d59154a9d9648ad8a04c0c637c2cb9b'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:46.810715 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:46.815392 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1411/2836] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:04:46 2022] DELETE /identity/v3/users/6d59154a9d9648ad8a04c0c637c2cb9b => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:04:47.223093 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534722dc-620f-433f-89df-015526368f8a admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:47.224301 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-534722dc-620f-433f-89df-015526368f8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:47.238925 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534722dc-620f-433f-89df-015526368f8a admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:47.309941 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa57fe8-c547-4082-a3fc-9fb1cdbf108f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:47.310966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa57fe8-c547-4082-a3fc-9fb1cdbf108f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:47.311490 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa57fe8-c547-4082-a3fc-9fb1cdbf108f admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:47.312116 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa57fe8-c547-4082-a3fc-9fb1cdbf108f admin admin] PATH_INFO: `/v3/projects/a4e7a48602b44e33b84855f19be1cce2` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:47.313897 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa57fe8-c547-4082-a3fc-9fb1cdbf108f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4e7a48602b44e33b84855f19be1cce2)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:47.316362 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa57fe8-c547-4082-a3fc-9fb1cdbf108f admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:47.319169 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:47.444254 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-4fa57fe8-c547-4082-a3fc-9fb1cdbf108f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4e7a48602b44e33b84855f19be1cce2', 'request_id': 'req-4fa57fe8-c547-4082-a3fc-9fb1cdbf108f'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:47.518770 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-4fa57fe8-c547-4082-a3fc-9fb1cdbf108f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4e7a48602b44e33b84855f19be1cce2', 'request_id': 'req-4fa57fe8-c547-4082-a3fc-9fb1cdbf108f'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:47.525059 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-4fa57fe8-c547-4082-a3fc-9fb1cdbf108f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4e7a48602b44e33b84855f19be1cce2', 'request_id': 'req-4fa57fe8-c547-4082-a3fc-9fb1cdbf108f'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:47.537855 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-4fa57fe8-c547-4082-a3fc-9fb1cdbf108f admin admin] The token cache is being invalidate because project a4e7a48602b44e33b84855f19be1cce2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:04:47.538308 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-4fa57fe8-c547-4082-a3fc-9fb1cdbf108f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:47.538786 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-4fa57fe8-c547-4082-a3fc-9fb1cdbf108f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:47.540346 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1426/2837] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:04:47 2022] DELETE /identity/v3/projects/a4e7a48602b44e33b84855f19be1cce2 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:04:47.544715 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f467b-f07a-46ed-b0a8-e24444be5c35 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:47.546172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-305f467b-f07a-46ed-b0a8-e24444be5c35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:47.560212 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305f467b-f07a-46ed-b0a8-e24444be5c35 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:47.618929 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a93f2-8db1-4dd0-82c5-bae6be1070af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:47.619605 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a93f2-8db1-4dd0-82c5-bae6be1070af admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:47.619887 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a93f2-8db1-4dd0-82c5-bae6be1070af admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:47.620169 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a93f2-8db1-4dd0-82c5-bae6be1070af admin admin] PATH_INFO: `/v3/users/edbe62909b0e42928e27a32e28ffc67a` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:47.621345 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a93f2-8db1-4dd0-82c5-bae6be1070af admin admin] RBAC: Authorizing `identity:delete_user(user_id=edbe62909b0e42928e27a32e28ffc67a)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:47.623305 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490a93f2-8db1-4dd0-82c5-bae6be1070af admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:47.839615 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-490a93f2-8db1-4dd0-82c5-bae6be1070af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edbe62909b0e42928e27a32e28ffc67a'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:47.840208 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-490a93f2-8db1-4dd0-82c5-bae6be1070af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edbe62909b0e42928e27a32e28ffc67a'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:47.909217 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-490a93f2-8db1-4dd0-82c5-bae6be1070af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edbe62909b0e42928e27a32e28ffc67a'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:47.916698 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-490a93f2-8db1-4dd0-82c5-bae6be1070af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edbe62909b0e42928e27a32e28ffc67a'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:47.928827 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-490a93f2-8db1-4dd0-82c5-bae6be1070af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edbe62909b0e42928e27a32e28ffc67a'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:47.934777 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1412/2838] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:04:47 2022] DELETE /identity/v3/users/edbe62909b0e42928e27a32e28ffc67a => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:04:48.339492 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa57fe8-c547-4082-a3fc-9fb1cdbf108f admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:48.340966 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-4fa57fe8-c547-4082-a3fc-9fb1cdbf108f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:48.354077 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa57fe8-c547-4082-a3fc-9fb1cdbf108f admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:48.419433 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949c9fe3-6804-4918-b672-1c75f1866851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:48.420073 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c9fe3-6804-4918-b672-1c75f1866851 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:48.420379 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c9fe3-6804-4918-b672-1c75f1866851 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:48.420652 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949c9fe3-6804-4918-b672-1c75f1866851 admin admin] PATH_INFO: `/v3/projects/bb47c2435df64f84bc22c7b0578ce711` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:48.421914 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949c9fe3-6804-4918-b672-1c75f1866851 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb47c2435df64f84bc22c7b0578ce711)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:48.423974 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949c9fe3-6804-4918-b672-1c75f1866851 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:48.540665 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-949c9fe3-6804-4918-b672-1c75f1866851 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb47c2435df64f84bc22c7b0578ce711', 'request_id': 'req-949c9fe3-6804-4918-b672-1c75f1866851'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:48.611742 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-949c9fe3-6804-4918-b672-1c75f1866851 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb47c2435df64f84bc22c7b0578ce711', 'request_id': 'req-949c9fe3-6804-4918-b672-1c75f1866851'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:48.618665 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-949c9fe3-6804-4918-b672-1c75f1866851 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb47c2435df64f84bc22c7b0578ce711', 'request_id': 'req-949c9fe3-6804-4918-b672-1c75f1866851'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:48.630670 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-949c9fe3-6804-4918-b672-1c75f1866851 admin admin] The token cache is being invalidate because project bb47c2435df64f84bc22c7b0578ce711 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:04:48.630956 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-949c9fe3-6804-4918-b672-1c75f1866851 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:48.631633 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-949c9fe3-6804-4918-b672-1c75f1866851 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:48.633221 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1427/2839] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:04:48 2022] DELETE /identity/v3/projects/bb47c2435df64f84bc22c7b0578ce711 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:04:48.638172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09dbf54-ffdc-4e68-a2de-7c440d4f9c01 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:48.638656 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09dbf54-ffdc-4e68-a2de-7c440d4f9c01 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:48.638780 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09dbf54-ffdc-4e68-a2de-7c440d4f9c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:48.658722 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-b09dbf54-ffdc-4e68-a2de-7c440d4f9c01 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:48.660730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-b09dbf54-ffdc-4e68-a2de-7c440d4f9c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:48.719923 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1413/2840] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:04:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:04:48.729720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994468c-9d2e-44ea-b552-306d25f4a8f0 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:48.734896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994468c-9d2e-44ea-b552-306d25f4a8f0 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:48.734896 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6994468c-9d2e-44ea-b552-306d25f4a8f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:48.750281 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-6994468c-9d2e-44ea-b552-306d25f4a8f0 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:48.751623 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-6994468c-9d2e-44ea-b552-306d25f4a8f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:48.759829 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1428/2841] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:04:48 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:04:48.764888 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09dbf54-ffdc-4e68-a2de-7c440d4f9c01 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:48.775207 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09dbf54-ffdc-4e68-a2de-7c440d4f9c01 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:48.779816 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba15367c-3cd3-4290-891b-3fbf119c4b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:48.780415 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15367c-3cd3-4290-891b-3fbf119c4b52 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:48.780698 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15367c-3cd3-4290-891b-3fbf119c4b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:48.780971 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba15367c-3cd3-4290-891b-3fbf119c4b52 admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:48.781695 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba15367c-3cd3-4290-891b-3fbf119c4b52 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:04:48.782001 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba15367c-3cd3-4290-891b-3fbf119c4b52 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:48.783625 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba15367c-3cd3-4290-891b-3fbf119c4b52 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:48.796551 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1414/2842] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 11:04:48 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 02 11:04:48.801340 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994468c-9d2e-44ea-b552-306d25f4a8f0 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:48.810528 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6994468c-9d2e-44ea-b552-306d25f4a8f0 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:48.815224 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e53e0-bd00-4236-b00f-f018bd780707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:48.821056 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e53e0-bd00-4236-b00f-f018bd780707 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:48.821056 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e53e0-bd00-4236-b00f-f018bd780707 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:48.821056 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e53e0-bd00-4236-b00f-f018bd780707 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:48.821056 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e53e0-bd00-4236-b00f-f018bd780707 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:48.821056 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e53e0-bd00-4236-b00f-f018bd780707 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:48.834563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:48.961083 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1429/2843] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:04:48 2022] POST /identity/v3/projects => generated 400 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:04:48.965216 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba15367c-3cd3-4290-891b-3fbf119c4b52 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:48.976051 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba15367c-3cd3-4290-891b-3fbf119c4b52 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:48.979885 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:49.031156 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f3274-9eb4-45f5-8219-7ab57fd78c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:49.031770 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f3274-9eb4-45f5-8219-7ab57fd78c1e admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:49.032048 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f3274-9eb4-45f5-8219-7ab57fd78c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:49.032320 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f3274-9eb4-45f5-8219-7ab57fd78c1e admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:49.032981 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f3274-9eb4-45f5-8219-7ab57fd78c1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:49.035861 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f3274-9eb4-45f5-8219-7ab57fd78c1e admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:49.116805 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1415/2844] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:04:48 2022] POST /identity/v3/users => generated 301 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:04:49.121042 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e53e0-bd00-4236-b00f-f018bd780707 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:49.132546 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e53e0-bd00-4236-b00f-f018bd780707 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:49.137400 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd946bc-1825-4df1-87dd-2be0af036cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:49.138048 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd946bc-1825-4df1-87dd-2be0af036cbf admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:49.138325 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd946bc-1825-4df1-87dd-2be0af036cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:49.138624 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd946bc-1825-4df1-87dd-2be0af036cbf admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:49.139347 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd946bc-1825-4df1-87dd-2be0af036cbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:49.140938 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd946bc-1825-4df1-87dd-2be0af036cbf admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:49.152021 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1430/2845] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:04:49 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:04:49.156040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f3274-9eb4-45f5-8219-7ab57fd78c1e admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:49.166319 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f3274-9eb4-45f5-8219-7ab57fd78c1e admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:49.171109 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509cb36f-084e-4a12-b9a9-395277409faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:49.171754 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509cb36f-084e-4a12-b9a9-395277409faf admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:49.172039 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509cb36f-084e-4a12-b9a9-395277409faf admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:49.172317 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509cb36f-084e-4a12-b9a9-395277409faf admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:49.173056 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509cb36f-084e-4a12-b9a9-395277409faf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:49.174691 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509cb36f-084e-4a12-b9a9-395277409faf admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:49.186411 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1416/2846] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:04:49 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:04:49.191008 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd946bc-1825-4df1-87dd-2be0af036cbf admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:49.201053 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd946bc-1825-4df1-87dd-2be0af036cbf admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:49.205671 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd0579-8135-486b-89ec-5610e8bf1dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:49.207170 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd0579-8135-486b-89ec-5610e8bf1dee admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:49.207563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd0579-8135-486b-89ec-5610e8bf1dee admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:49.207853 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd0579-8135-486b-89ec-5610e8bf1dee admin admin] PATH_INFO: `/v3/projects/cccfce71f6454fb2981b6109f049f7ea/users/d01d01bdf99f4b238d71a4fb11d5f0b8/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:49.224795 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd0579-8135-486b-89ec-5610e8bf1dee admin admin] RBAC: Authorizing `identity:create_grant(project_id=cccfce71f6454fb2981b6109f049f7ea, user_id=d01d01bdf99f4b238d71a4fb11d5f0b8, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:49.227413 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd0579-8135-486b-89ec-5610e8bf1dee admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:49.282420 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1431/2847] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:04:49 2022] PUT /identity/v3/projects/cccfce71f6454fb2981b6109f049f7ea/users/d01d01bdf99f4b238d71a4fb11d5f0b8/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:04:49.296059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91336eaa-8b4e-4691-b22b-2ed74f09da20 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:49.296059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91336eaa-8b4e-4691-b22b-2ed74f09da20 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:49.296059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91336eaa-8b4e-4691-b22b-2ed74f09da20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:49.324858 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-91336eaa-8b4e-4691-b22b-2ed74f09da20 None None] MFA Rules not processed for user `d01d01bdf99f4b238d71a4fb11d5f0b8`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:49.326215 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-91336eaa-8b4e-4691-b22b-2ed74f09da20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:49.395513 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1417/2848] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:04:49 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:04:49.406962 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd0579-8135-486b-89ec-5610e8bf1dee admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:49.408668 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-05bd0579-8135-486b-89ec-5610e8bf1dee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:49.482170 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd0579-8135-486b-89ec-5610e8bf1dee admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:49.485721 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379ba00-4d92-4687-be1d-23d0709203db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc9ab3c9075b4a8385ef634dc0b1625e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:49.486369 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379ba00-4d92-4687-be1d-23d0709203db service cinder] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:49.486718 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379ba00-4d92-4687-be1d-23d0709203db service cinder] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:49.487003 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6379ba00-4d92-4687-be1d-23d0709203db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:49.495414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379ba00-4d92-4687-be1d-23d0709203db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:49.496839 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6379ba00-4d92-4687-be1d-23d0709203db service cinder] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:49.505608 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1432/2849] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:04:49 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:04:56.814142 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91336eaa-8b4e-4691-b22b-2ed74f09da20 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:56.815603 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-91336eaa-8b4e-4691-b22b-2ed74f09da20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:56.875275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91336eaa-8b4e-4691-b22b-2ed74f09da20 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:56.878807 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97025d0-c0c2-4039-8c73-1ce944df10cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:56.879724 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97025d0-c0c2-4039-8c73-1ce944df10cf service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:56.880016 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97025d0-c0c2-4039-8c73-1ce944df10cf service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:56.880289 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97025d0-c0c2-4039-8c73-1ce944df10cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:56.882040 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-a97025d0-c0c2-4039-8c73-1ce944df10cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:56.913246 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:56.966266 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97025d0-c0c2-4039-8c73-1ce944df10cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:56.967775 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97025d0-c0c2-4039-8c73-1ce944df10cf service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:56.990155 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1418/2850] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:04:56 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:04:58.174913 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379ba00-4d92-4687-be1d-23d0709203db service cinder] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:58.185054 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6379ba00-4d92-4687-be1d-23d0709203db service cinder] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:58.189216 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1579b72a-a61b-49fb-b445-f50d45492d5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:58.189827 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579b72a-a61b-49fb-b445-f50d45492d5e service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:58.190107 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579b72a-a61b-49fb-b445-f50d45492d5e service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:58.190407 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579b72a-a61b-49fb-b445-f50d45492d5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:58.192416 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-1579b72a-a61b-49fb-b445-f50d45492d5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:58.206013 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1579b72a-a61b-49fb-b445-f50d45492d5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:58.210373 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1579b72a-a61b-49fb-b445-f50d45492d5e service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:58.218674 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1433/2851] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:04:58 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:04:58.476563 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97025d0-c0c2-4039-8c73-1ce944df10cf service neutron] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:58.542547 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97025d0-c0c2-4039-8c73-1ce944df10cf service neutron] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:58.547366 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f1c91-9f1b-4845-98fb-041cadd45b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:58.547766 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f1c91-9f1b-4845-98fb-041cadd45b0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:58.548097 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f1c91-9f1b-4845-98fb-041cadd45b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:58.548277 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f1c91-9f1b-4845-98fb-041cadd45b0f admin admin] PATH_INFO: `/v3/users/d01d01bdf99f4b238d71a4fb11d5f0b8` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:58.549443 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f1c91-9f1b-4845-98fb-041cadd45b0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d01d01bdf99f4b238d71a4fb11d5f0b8)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:58.551395 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f1c91-9f1b-4845-98fb-041cadd45b0f admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:58.701420 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-aa1f1c91-9f1b-4845-98fb-041cadd45b0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd01d01bdf99f4b238d71a4fb11d5f0b8'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:58.702166 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-aa1f1c91-9f1b-4845-98fb-041cadd45b0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd01d01bdf99f4b238d71a4fb11d5f0b8'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:58.760195 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-aa1f1c91-9f1b-4845-98fb-041cadd45b0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd01d01bdf99f4b238d71a4fb11d5f0b8'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:58.768377 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-aa1f1c91-9f1b-4845-98fb-041cadd45b0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd01d01bdf99f4b238d71a4fb11d5f0b8'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:58.781593 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-aa1f1c91-9f1b-4845-98fb-041cadd45b0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd01d01bdf99f4b238d71a4fb11d5f0b8'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:58.787887 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1419/2852] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:04:58 2022] DELETE /identity/v3/users/d01d01bdf99f4b238d71a4fb11d5f0b8 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:04:58.792510 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181caa5-1b1f-4694-bbce-ae63acbf4858 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:58.792866 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181caa5-1b1f-4694-bbce-ae63acbf4858 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:58.793277 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181caa5-1b1f-4694-bbce-ae63acbf4858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:58.813563 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-f181caa5-1b1f-4694-bbce-ae63acbf4858 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:04:58.815374 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f181caa5-1b1f-4694-bbce-ae63acbf4858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:58.876236 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1434/2853] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:04:58 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 02 11:04:58.886597 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f1c91-9f1b-4845-98fb-041cadd45b0f admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:58.888113 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-aa1f1c91-9f1b-4845-98fb-041cadd45b0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:58.907944 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:58.952254 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f1c91-9f1b-4845-98fb-041cadd45b0f admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:58.956036 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3846c947-10f7-47c1-81bb-26d401f44a60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:58.956659 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846c947-10f7-47c1-81bb-26d401f44a60 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:58.973277 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846c947-10f7-47c1-81bb-26d401f44a60 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:58.973277 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3846c947-10f7-47c1-81bb-26d401f44a60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:58.973277 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3846c947-10f7-47c1-81bb-26d401f44a60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:58.973277 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3846c947-10f7-47c1-81bb-26d401f44a60 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:58.982482 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1420/2854] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:04:58 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:04:59.501154 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181caa5-1b1f-4694-bbce-ae63acbf4858 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:04:59.504663 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-f181caa5-1b1f-4694-bbce-ae63acbf4858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:04:59.516713 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181caa5-1b1f-4694-bbce-ae63acbf4858 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:04:59.521779 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3964143-628d-437b-9b70-c8d753e056c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:04:59.522429 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3964143-628d-437b-9b70-c8d753e056c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:04:59.522778 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3964143-628d-437b-9b70-c8d753e056c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:04:59.522976 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3964143-628d-437b-9b70-c8d753e056c2 admin admin] PATH_INFO: `/v3/projects/cccfce71f6454fb2981b6109f049f7ea` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:04:59.525160 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3964143-628d-437b-9b70-c8d753e056c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cccfce71f6454fb2981b6109f049f7ea)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:04:59.527338 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3964143-628d-437b-9b70-c8d753e056c2 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:04:59.645120 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:04:59.654946 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-c3964143-628d-437b-9b70-c8d753e056c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cccfce71f6454fb2981b6109f049f7ea', 'request_id': 'req-c3964143-628d-437b-9b70-c8d753e056c2'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:59.756691 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-c3964143-628d-437b-9b70-c8d753e056c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cccfce71f6454fb2981b6109f049f7ea', 'request_id': 'req-c3964143-628d-437b-9b70-c8d753e056c2'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:59.763690 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-c3964143-628d-437b-9b70-c8d753e056c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cccfce71f6454fb2981b6109f049f7ea', 'request_id': 'req-c3964143-628d-437b-9b70-c8d753e056c2'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:59.775943 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-c3964143-628d-437b-9b70-c8d753e056c2 admin admin] The token cache is being invalidate because project cccfce71f6454fb2981b6109f049f7ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:04:59.780740 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-c3964143-628d-437b-9b70-c8d753e056c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:59.780740 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-c3964143-628d-437b-9b70-c8d753e056c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:04:59.780740 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1435/2855] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:04:59 2022] DELETE /identity/v3/projects/cccfce71f6454fb2981b6109f049f7ea => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:05:09.213624 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=19 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:05:09.845872 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:DELETE=5,keystone:SELECT=11,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:05:10.860572 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea233ddb-b9db-42fc-9058-24908c05560e None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:05:10.861126 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea233ddb-b9db-42fc-9058-24908c05560e None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:05:10.861126 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea233ddb-b9db-42fc-9058-24908c05560e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:05:10.879924 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-ea233ddb-b9db-42fc-9058-24908c05560e None None] MFA Rules not processed for user `6bf80e69ea524d83ab199fc7315f4be2`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:05:10.881122 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-ea233ddb-b9db-42fc-9058-24908c05560e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:05:10.936567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1421/2856] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:05:10 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:05:20.930263 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=25 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:05:23.115694 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3964143-628d-437b-9b70-c8d753e056c2 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:05:23.117606 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-c3964143-628d-437b-9b70-c8d753e056c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:05:23.177610 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3964143-628d-437b-9b70-c8d753e056c2 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:05:23.181925 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e52ac3-c6dc-446f-89bd-ffc086201566 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:05:23.182540 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e52ac3-c6dc-446f-89bd-ffc086201566 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:05:23.182816 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e52ac3-c6dc-446f-89bd-ffc086201566 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:05:23.183121 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e52ac3-c6dc-446f-89bd-ffc086201566 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:05:23.184915 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-d2e52ac3-c6dc-446f-89bd-ffc086201566 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:05:23.195998 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e52ac3-c6dc-446f-89bd-ffc086201566 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:05:23.197439 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e52ac3-c6dc-446f-89bd-ffc086201566 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:05:23.206267 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1436/2857] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:05:23 2022] GET /identity/v3/auth/tokens => generated 3131 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:05:30.704669 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea233ddb-b9db-42fc-9058-24908c05560e None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:05:30.717748 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea233ddb-b9db-42fc-9058-24908c05560e None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:05:30.721354 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023534da-b25c-42c2-a101-5b37afdf7eed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:05:30.721975 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023534da-b25c-42c2-a101-5b37afdf7eed service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:05:30.722256 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023534da-b25c-42c2-a101-5b37afdf7eed service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:05:30.722531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023534da-b25c-42c2-a101-5b37afdf7eed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:05:30.724362 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-023534da-b25c-42c2-a101-5b37afdf7eed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:05:30.796302 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023534da-b25c-42c2-a101-5b37afdf7eed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:05:30.797746 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023534da-b25c-42c2-a101-5b37afdf7eed service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:05:30.808834 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1422/2858] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:05:30 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:05:33.202139 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=26 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:05:38.464438 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e52ac3-c6dc-446f-89bd-ffc086201566 service neutron] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:05:38.466413 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-d2e52ac3-c6dc-446f-89bd-ffc086201566 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:05:38.481417 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e52ac3-c6dc-446f-89bd-ffc086201566 service neutron] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:05:38.486095 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ca374-d469-48fb-ad4e-96e740ac50a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:05:38.486725 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ca374-d469-48fb-ad4e-96e740ac50a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:05:38.487028 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ca374-d469-48fb-ad4e-96e740ac50a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:05:38.487320 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ca374-d469-48fb-ad4e-96e740ac50a9 admin admin] PATH_INFO: `/v3/users/914de37664914b4cb91e0ce2c3a0e1d1` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:05:38.488698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ca374-d469-48fb-ad4e-96e740ac50a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=914de37664914b4cb91e0ce2c3a0e1d1)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:05:38.490383 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ca374-d469-48fb-ad4e-96e740ac50a9 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:05:38.670189 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-114ca374-d469-48fb-ad4e-96e740ac50a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '914de37664914b4cb91e0ce2c3a0e1d1'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:05:38.678077 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-114ca374-d469-48fb-ad4e-96e740ac50a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '914de37664914b4cb91e0ce2c3a0e1d1'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:05:38.679515 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-114ca374-d469-48fb-ad4e-96e740ac50a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '914de37664914b4cb91e0ce2c3a0e1d1'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:05:38.770267 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-114ca374-d469-48fb-ad4e-96e740ac50a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '914de37664914b4cb91e0ce2c3a0e1d1'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:05:38.783853 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-114ca374-d469-48fb-ad4e-96e740ac50a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '914de37664914b4cb91e0ce2c3a0e1d1'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:05:38.789971 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1437/2859] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:05:38 2022] DELETE /identity/v3/users/914de37664914b4cb91e0ce2c3a0e1d1 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:05:39.036275 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023534da-b25c-42c2-a101-5b37afdf7eed service neutron] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:05:39.049088 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023534da-b25c-42c2-a101-5b37afdf7eed service neutron] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:05:39.110675 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ce01c-0c28-4854-be8c-ae77cc36c980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:05:39.111358 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ce01c-0c28-4854-be8c-ae77cc36c980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:05:39.111686 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ce01c-0c28-4854-be8c-ae77cc36c980 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:05:39.111985 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237ce01c-0c28-4854-be8c-ae77cc36c980 admin admin] PATH_INFO: `/v3/projects/be66234794284419a47e3afcae968084` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:05:39.113164 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ce01c-0c28-4854-be8c-ae77cc36c980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be66234794284419a47e3afcae968084)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:05:39.115102 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237ce01c-0c28-4854-be8c-ae77cc36c980 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:05:39.260742 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-237ce01c-0c28-4854-be8c-ae77cc36c980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be66234794284419a47e3afcae968084', 'request_id': 'req-237ce01c-0c28-4854-be8c-ae77cc36c980'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:05:39.267436 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-237ce01c-0c28-4854-be8c-ae77cc36c980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be66234794284419a47e3afcae968084', 'request_id': 'req-237ce01c-0c28-4854-be8c-ae77cc36c980'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:05:39.348946 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-237ce01c-0c28-4854-be8c-ae77cc36c980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be66234794284419a47e3afcae968084', 'request_id': 'req-237ce01c-0c28-4854-be8c-ae77cc36c980'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:05:39.360277 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-237ce01c-0c28-4854-be8c-ae77cc36c980 admin admin] The token cache is being invalidate because project be66234794284419a47e3afcae968084 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:05:39.360563 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-237ce01c-0c28-4854-be8c-ae77cc36c980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:05:39.361187 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-237ce01c-0c28-4854-be8c-ae77cc36c980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:05:39.362687 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1423/2860] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:05:39 2022] DELETE /identity/v3/projects/be66234794284419a47e3afcae968084 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:05:39.367022 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ca374-d469-48fb-ad4e-96e740ac50a9 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:05:39.368520 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-114ca374-d469-48fb-ad4e-96e740ac50a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:05:39.382318 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ca374-d469-48fb-ad4e-96e740ac50a9 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:05:39.453070 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca2f88-6cf7-45a9-aa1a-383416da07ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:05:39.453715 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca2f88-6cf7-45a9-aa1a-383416da07ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:05:39.453998 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca2f88-6cf7-45a9-aa1a-383416da07ff admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:05:39.454275 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca2f88-6cf7-45a9-aa1a-383416da07ff admin admin] PATH_INFO: `/v3/users/95c028783b47498f817dbfc51e7958a0` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:05:39.455500 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca2f88-6cf7-45a9-aa1a-383416da07ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=95c028783b47498f817dbfc51e7958a0)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:05:39.457500 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca2f88-6cf7-45a9-aa1a-383416da07ff admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:05:39.671358 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-03ca2f88-6cf7-45a9-aa1a-383416da07ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95c028783b47498f817dbfc51e7958a0'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:05:39.678267 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-03ca2f88-6cf7-45a9-aa1a-383416da07ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95c028783b47498f817dbfc51e7958a0'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:05:39.679454 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-03ca2f88-6cf7-45a9-aa1a-383416da07ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95c028783b47498f817dbfc51e7958a0'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:05:39.759502 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-03ca2f88-6cf7-45a9-aa1a-383416da07ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95c028783b47498f817dbfc51e7958a0'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:05:39.768425 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:05:39.773226 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-03ca2f88-6cf7-45a9-aa1a-383416da07ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95c028783b47498f817dbfc51e7958a0'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:05:39.779551 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1438/2861] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:05:39 2022] DELETE /identity/v3/users/95c028783b47498f817dbfc51e7958a0 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:05:40.008312 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ce01c-0c28-4854-be8c-ae77cc36c980 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:05:40.011180 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-237ce01c-0c28-4854-be8c-ae77cc36c980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:05:40.024297 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237ce01c-0c28-4854-be8c-ae77cc36c980 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:05:40.046593 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:05:40.084309 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416d106-53b7-491b-9c3b-0681b840e46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:05:40.084956 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416d106-53b7-491b-9c3b-0681b840e46d admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:05:40.085293 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416d106-53b7-491b-9c3b-0681b840e46d admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:05:40.085576 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3416d106-53b7-491b-9c3b-0681b840e46d admin admin] PATH_INFO: `/v3/projects/07312de08a584c85a9e20a1da10d58ed` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:05:40.086765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416d106-53b7-491b-9c3b-0681b840e46d admin admin] RBAC: Authorizing `identity:delete_project(project_id=07312de08a584c85a9e20a1da10d58ed)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:05:40.088730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3416d106-53b7-491b-9c3b-0681b840e46d admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:05:40.198571 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-3416d106-53b7-491b-9c3b-0681b840e46d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07312de08a584c85a9e20a1da10d58ed', 'request_id': 'req-3416d106-53b7-491b-9c3b-0681b840e46d'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:05:40.204810 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-3416d106-53b7-491b-9c3b-0681b840e46d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07312de08a584c85a9e20a1da10d58ed', 'request_id': 'req-3416d106-53b7-491b-9c3b-0681b840e46d'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:05:40.284305 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-3416d106-53b7-491b-9c3b-0681b840e46d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07312de08a584c85a9e20a1da10d58ed', 'request_id': 'req-3416d106-53b7-491b-9c3b-0681b840e46d'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:05:40.296514 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-3416d106-53b7-491b-9c3b-0681b840e46d admin admin] The token cache is being invalidate because project 07312de08a584c85a9e20a1da10d58ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:05:40.296799 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-3416d106-53b7-491b-9c3b-0681b840e46d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:05:40.297435 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-3416d106-53b7-491b-9c3b-0681b840e46d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:05:40.298945 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1424/2862] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:05:40 2022] DELETE /identity/v3/projects/07312de08a584c85a9e20a1da10d58ed => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:05:49.892284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=3,keystone:DELETE=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:05:50.293172 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:05:51.106898 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca2f88-6cf7-45a9-aa1a-383416da07ff admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:05:51.108485 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-03ca2f88-6cf7-45a9-aa1a-383416da07ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:05:51.182367 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca2f88-6cf7-45a9-aa1a-383416da07ff admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:05:51.244039 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd2c443-b4b2-42cd-81ec-4c019385e428 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:05:51.244659 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd2c443-b4b2-42cd-81ec-4c019385e428 service nova] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:05:51.244934 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd2c443-b4b2-42cd-81ec-4c019385e428 service nova] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:05:51.245211 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd2c443-b4b2-42cd-81ec-4c019385e428 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:05:51.247163 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-8cd2c443-b4b2-42cd-81ec-4c019385e428 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:05:51.260085 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd2c443-b4b2-42cd-81ec-4c019385e428 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:05:51.261697 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd2c443-b4b2-42cd-81ec-4c019385e428 service nova] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:05:51.273088 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1439/2863] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:05:51 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:06:01.269072 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=62 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:06:17.936100 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416d106-53b7-491b-9c3b-0681b840e46d admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:06:17.938837 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-3416d106-53b7-491b-9c3b-0681b840e46d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:06:18.004111 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3416d106-53b7-491b-9c3b-0681b840e46d admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:06:18.008653 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b140f-1e48-46dc-825a-d1aad151b578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:06:18.009279 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b140f-1e48-46dc-825a-d1aad151b578 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:06:18.009554 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b140f-1e48-46dc-825a-d1aad151b578 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:06:18.009825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b140f-1e48-46dc-825a-d1aad151b578 admin admin] PATH_INFO: `/v3/users/6bf80e69ea524d83ab199fc7315f4be2` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:06:18.011081 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b140f-1e48-46dc-825a-d1aad151b578 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bf80e69ea524d83ab199fc7315f4be2)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:06:18.013624 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b140f-1e48-46dc-825a-d1aad151b578 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:06:18.177001 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-993b140f-1e48-46dc-825a-d1aad151b578 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bf80e69ea524d83ab199fc7315f4be2'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:06:18.177604 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-993b140f-1e48-46dc-825a-d1aad151b578 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bf80e69ea524d83ab199fc7315f4be2'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:06:18.249427 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-993b140f-1e48-46dc-825a-d1aad151b578 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bf80e69ea524d83ab199fc7315f4be2'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:06:18.257178 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-993b140f-1e48-46dc-825a-d1aad151b578 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bf80e69ea524d83ab199fc7315f4be2'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:06:18.270501 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-993b140f-1e48-46dc-825a-d1aad151b578 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bf80e69ea524d83ab199fc7315f4be2'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:06:18.277014 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1425/2864] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:06:17 2022] DELETE /identity/v3/users/6bf80e69ea524d83ab199fc7315f4be2 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:06:18.521292 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd2c443-b4b2-42cd-81ec-4c019385e428 service nova] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:06:18.537004 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd2c443-b4b2-42cd-81ec-4c019385e428 service nova] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:06:18.590314 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be36843-8415-4391-9c83-5c5dc81dba3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:06:18.590955 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be36843-8415-4391-9c83-5c5dc81dba3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:06:18.591289 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be36843-8415-4391-9c83-5c5dc81dba3c admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:06:18.591603 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be36843-8415-4391-9c83-5c5dc81dba3c admin admin] PATH_INFO: `/v3/projects/1ebdffccaaa04484b32602a67fe23629` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:06:18.592768 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be36843-8415-4391-9c83-5c5dc81dba3c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ebdffccaaa04484b32602a67fe23629)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:06:18.594702 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be36843-8415-4391-9c83-5c5dc81dba3c admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:06:18.695929 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2be36843-8415-4391-9c83-5c5dc81dba3c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ebdffccaaa04484b32602a67fe23629', 'request_id': 'req-2be36843-8415-4391-9c83-5c5dc81dba3c'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:06:18.781674 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2be36843-8415-4391-9c83-5c5dc81dba3c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ebdffccaaa04484b32602a67fe23629', 'request_id': 'req-2be36843-8415-4391-9c83-5c5dc81dba3c'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:06:18.787641 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2be36843-8415-4391-9c83-5c5dc81dba3c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ebdffccaaa04484b32602a67fe23629', 'request_id': 'req-2be36843-8415-4391-9c83-5c5dc81dba3c'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:06:18.799701 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2be36843-8415-4391-9c83-5c5dc81dba3c admin admin] The token cache is being invalidate because project 1ebdffccaaa04484b32602a67fe23629 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92111) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:06:18.799991 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2be36843-8415-4391-9c83-5c5dc81dba3c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:06:18.800633 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-2be36843-8415-4391-9c83-5c5dc81dba3c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:06:18.802159 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1440/2865] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:06:18 2022] DELETE /identity/v3/projects/1ebdffccaaa04484b32602a67fe23629 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:06:18.807183 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54358032-dd5a-49c5-a30b-6a151b8acae3 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:06:18.807500 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54358032-dd5a-49c5-a30b-6a151b8acae3 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:06:18.807777 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54358032-dd5a-49c5-a30b-6a151b8acae3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:06:18.827533 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-54358032-dd5a-49c5-a30b-6a151b8acae3 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:06:18.828729 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-54358032-dd5a-49c5-a30b-6a151b8acae3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:06:18.887370 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1426/2866] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:06:18 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:06:18.895481 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96a88c-c4ff-44e0-92af-c2dfa2f59ecd None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:06:18.895772 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96a88c-c4ff-44e0-92af-c2dfa2f59ecd None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:06:18.896050 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af96a88c-c4ff-44e0-92af-c2dfa2f59ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:06:18.915703 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-af96a88c-c4ff-44e0-92af-c2dfa2f59ecd None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:06:18.916960 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-af96a88c-c4ff-44e0-92af-c2dfa2f59ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:06:18.925364 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1441/2867] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:06:18 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:06:18.929475 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54358032-dd5a-49c5-a30b-6a151b8acae3 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:06:18.939947 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54358032-dd5a-49c5-a30b-6a151b8acae3 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:06:18.944522 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c11d1-d26c-4e04-b5fa-9ea5fcacbe63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:06:18.945123 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c11d1-d26c-4e04-b5fa-9ea5fcacbe63 admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:06:18.945410 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c11d1-d26c-4e04-b5fa-9ea5fcacbe63 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:06:18.945690 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438c11d1-d26c-4e04-b5fa-9ea5fcacbe63 admin admin] PATH_INFO: `/v3/domains` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:06:18.946383 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c11d1-d26c-4e04-b5fa-9ea5fcacbe63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=92110) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 02 11:06:18.946689 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c11d1-d26c-4e04-b5fa-9ea5fcacbe63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:06:18.948445 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438c11d1-d26c-4e04-b5fa-9ea5fcacbe63 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:06:18.959981 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1427/2868] 10.222.0.115 () {58 vars in 1158 bytes} [Tue Aug 2 11:06:18 2022] GET /identity/v3/domains?name=Default => generated 310 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 02 11:06:18.964456 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96a88c-c4ff-44e0-92af-c2dfa2f59ecd None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:06:18.973741 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af96a88c-c4ff-44e0-92af-c2dfa2f59ecd None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:06:18.978376 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2fd4a-c7f5-40f3-8b84-3bb1526bfa19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:06:18.978976 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2fd4a-c7f5-40f3-8b84-3bb1526bfa19 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:06:18.979284 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2fd4a-c7f5-40f3-8b84-3bb1526bfa19 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:06:18.979559 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2fd4a-c7f5-40f3-8b84-3bb1526bfa19 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:06:18.980733 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2fd4a-c7f5-40f3-8b84-3bb1526bfa19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:06:18.982673 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b2fd4a-c7f5-40f3-8b84-3bb1526bfa19 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:06:19.084856 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1442/2869] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:06:18 2022] POST /identity/v3/projects => generated 384 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 02 11:06:19.101919 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c11d1-d26c-4e04-b5fa-9ea5fcacbe63 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:06:19.112740 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438c11d1-d26c-4e04-b5fa-9ea5fcacbe63 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:06:19.116351 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:06:19.179164 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c633e8b-24ab-4323-8a9c-d72244ed1517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:06:19.179782 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c633e8b-24ab-4323-8a9c-d72244ed1517 admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:06:19.180068 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c633e8b-24ab-4323-8a9c-d72244ed1517 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:06:19.180347 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c633e8b-24ab-4323-8a9c-d72244ed1517 admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:06:19.181020 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c633e8b-24ab-4323-8a9c-d72244ed1517 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:06:19.183002 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c633e8b-24ab-4323-8a9c-d72244ed1517 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:06:19.280059 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1428/2870] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:06:19 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:06:19.284280 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2fd4a-c7f5-40f3-8b84-3bb1526bfa19 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:06:19.299924 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2fd4a-c7f5-40f3-8b84-3bb1526bfa19 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:06:19.304562 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a972446a-c776-408b-ab96-1e07f19e70e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:06:19.305212 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972446a-c776-408b-ab96-1e07f19e70e7 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:06:19.305487 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972446a-c776-408b-ab96-1e07f19e70e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:06:19.305755 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a972446a-c776-408b-ab96-1e07f19e70e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:06:19.306435 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a972446a-c776-408b-ab96-1e07f19e70e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:06:19.308081 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a972446a-c776-408b-ab96-1e07f19e70e7 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:06:19.319409 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1443/2871] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:06:19 2022] GET /identity/v3/roles => generated 1427 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:06:19.324282 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c633e8b-24ab-4323-8a9c-d72244ed1517 admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:06:19.334848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c633e8b-24ab-4323-8a9c-d72244ed1517 admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:06:19.341733 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d5c7aa-d608-4e29-b10d-6950ebd654fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:06:19.342284 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5c7aa-d608-4e29-b10d-6950ebd654fe admin admin] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:06:19.342567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5c7aa-d608-4e29-b10d-6950ebd654fe admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:06:19.342845 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d5c7aa-d608-4e29-b10d-6950ebd654fe admin admin] PATH_INFO: `/v3/roles` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:06:19.343642 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d5c7aa-d608-4e29-b10d-6950ebd654fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:06:19.345245 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d5c7aa-d608-4e29-b10d-6950ebd654fe admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:06:19.356825 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1429/2872] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:06:19 2022] GET /identity/v3/roles => generated 1427 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:06:19.361323 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a972446a-c776-408b-ab96-1e07f19e70e7 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:06:19.371337 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a972446a-c776-408b-ab96-1e07f19e70e7 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:06:19.376005 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ccbe28-300f-409b-8f7d-ef236e83e33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:06:19.376657 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ccbe28-300f-409b-8f7d-ef236e83e33a admin admin] REQUEST_METHOD: `PUT` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:06:19.376935 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ccbe28-300f-409b-8f7d-ef236e83e33a admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:06:19.377208 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ccbe28-300f-409b-8f7d-ef236e83e33a admin admin] PATH_INFO: `/v3/projects/bb052a6c83084dd9bdbf408ad8870c16/users/7509ddedcec74149bb850e4adf64b489/roles/33c5141e078844bbb60127f2b26540d7` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:06:19.391836 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ccbe28-300f-409b-8f7d-ef236e83e33a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb052a6c83084dd9bdbf408ad8870c16, user_id=7509ddedcec74149bb850e4adf64b489, role_id=33c5141e078844bbb60127f2b26540d7)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:06:19.394494 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ccbe28-300f-409b-8f7d-ef236e83e33a admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:06:19.464368 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1444/2873] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:06:19 2022] PUT /identity/v3/projects/bb052a6c83084dd9bdbf408ad8870c16/users/7509ddedcec74149bb850e4adf64b489/roles/33c5141e078844bbb60127f2b26540d7 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 02 11:06:19.470284 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd399a-a504-47d6-98ba-faf52a7f9208 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:06:19.470570 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd399a-a504-47d6-98ba-faf52a7f9208 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:06:19.470847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd399a-a504-47d6-98ba-faf52a7f9208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:06:19.490970 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-b5dd399a-a504-47d6-98ba-faf52a7f9208 None None] MFA Rules not processed for user `4fb0c392df554f5dbd68f6861be954a1`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:06:19.492199 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-b5dd399a-a504-47d6-98ba-faf52a7f9208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:06:19.552592 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1430/2874] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:06:19 2022] POST /identity/v3/auth/tokens => generated 3249 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:06:19.562311 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ccbe28-300f-409b-8f7d-ef236e83e33a admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:06:19.564055 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-37ccbe28-300f-409b-8f7d-ef236e83e33a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:06:19.596018 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:06:19.621421 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ccbe28-300f-409b-8f7d-ef236e83e33a admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:06:19.625824 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115fde4-c984-4154-b4be-2d78bc51aaa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:06:19.626440 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115fde4-c984-4154-b4be-2d78bc51aaa2 service neutron] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:06:19.626720 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115fde4-c984-4154-b4be-2d78bc51aaa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:06:19.626993 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f115fde4-c984-4154-b4be-2d78bc51aaa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:06:19.636059 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f115fde4-c984-4154-b4be-2d78bc51aaa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:06:19.638116 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f115fde4-c984-4154-b4be-2d78bc51aaa2 service neutron] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:06:19.647842 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1445/2875] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:06:19 2022] GET /identity/v3/auth/tokens => generated 3249 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:06:24.905955 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b5b3e-f307-4cf5-a253-85865846e354 None None] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:06:24.906494 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b5b3e-f307-4cf5-a253-85865846e354 None None] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:06:24.906494 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628b5b3e-f307-4cf5-a253-85865846e354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:06:24.940915 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.auth.core [None req-628b5b3e-f307-4cf5-a253-85865846e354 None None] MFA Rules not processed for user `7509ddedcec74149bb850e4adf64b489`. Rule list: `[]` (Enabled: `True`). {{(pid=92110) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:06:24.942240 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-628b5b3e-f307-4cf5-a253-85865846e354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:06:25.011677 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1431/2876] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:06:24 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:06:25.017789 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115fde4-c984-4154-b4be-2d78bc51aaa2 service neutron] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:06:25.028515 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f115fde4-c984-4154-b4be-2d78bc51aaa2 service neutron] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:06:25.033613 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed30299-b75c-4a81-ba40-045219888a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:06:25.034333 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed30299-b75c-4a81-ba40-045219888a23 admin admin] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:06:25.034693 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed30299-b75c-4a81-ba40-045219888a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:06:25.035040 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed30299-b75c-4a81-ba40-045219888a23 admin admin] PATH_INFO: `/v3/projects` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:06:25.035841 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed30299-b75c-4a81-ba40-045219888a23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:06:25.037874 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed30299-b75c-4a81-ba40-045219888a23 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:06:25.115339 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1446/2877] 10.222.0.115 () {60 vars in 1158 bytes} [Tue Aug 2 11:06:25 2022] POST /identity/v3/projects => generated 384 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:06:25.119344 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b5b3e-f307-4cf5-a253-85865846e354 None None] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:06:25.131699 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628b5b3e-f307-4cf5-a253-85865846e354 None None] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:06:25.149593 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:06:25.198148 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466aa015-022a-4ee8-a064-2b73b0df618d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:06:25.198777 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466aa015-022a-4ee8-a064-2b73b0df618d admin admin] REQUEST_METHOD: `POST` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:06:25.199083 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466aa015-022a-4ee8-a064-2b73b0df618d admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:06:25.200147 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466aa015-022a-4ee8-a064-2b73b0df618d admin admin] PATH_INFO: `/v3/users` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:06:25.201372 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466aa015-022a-4ee8-a064-2b73b0df618d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:06:25.202738 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466aa015-022a-4ee8-a064-2b73b0df618d admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:06:25.314737 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1432/2878] 10.222.0.115 () {60 vars in 1149 bytes} [Tue Aug 2 11:06:25 2022] POST /identity/v3/users => generated 299 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 02 11:06:25.317717 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed30299-b75c-4a81-ba40-045219888a23 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:06:25.328593 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed30299-b75c-4a81-ba40-045219888a23 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:06:25.333212 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc30c14-e73f-4d10-8b6a-627f24834a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:06:25.333833 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc30c14-e73f-4d10-8b6a-627f24834a05 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:06:25.334111 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc30c14-e73f-4d10-8b6a-627f24834a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:06:25.334417 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc30c14-e73f-4d10-8b6a-627f24834a05 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:06:25.335282 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc30c14-e73f-4d10-8b6a-627f24834a05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:06:25.336885 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc30c14-e73f-4d10-8b6a-627f24834a05 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:06:25.348571 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1447/2879] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:06:25 2022] GET /identity/v3/roles => generated 1427 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 02 11:06:25.352889 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466aa015-022a-4ee8-a064-2b73b0df618d admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:06:25.364353 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466aa015-022a-4ee8-a064-2b73b0df618d admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:06:25.369701 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f5466-5ac6-4035-9359-14c7c8f650dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:06:25.370369 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f5466-5ac6-4035-9359-14c7c8f650dd admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:06:25.370652 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f5466-5ac6-4035-9359-14c7c8f650dd admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:06:25.370928 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f5466-5ac6-4035-9359-14c7c8f650dd admin admin] PATH_INFO: `/v3/projects/fe82126e018746fe8b88a098d205057d/users/1be384e468c24d5e93a53139fb697ec8/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:06:25.387457 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f5466-5ac6-4035-9359-14c7c8f650dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe82126e018746fe8b88a098d205057d, user_id=1be384e468c24d5e93a53139fb697ec8, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:06:25.390595 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940f5466-5ac6-4035-9359-14c7c8f650dd admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:06:25.463781 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1433/2880] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:06:25 2022] PUT /identity/v3/projects/fe82126e018746fe8b88a098d205057d/users/1be384e468c24d5e93a53139fb697ec8/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:06:25.468476 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc30c14-e73f-4d10-8b6a-627f24834a05 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:06:25.479333 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc30c14-e73f-4d10-8b6a-627f24834a05 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:06:25.542304 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e6837-a097-4100-b3ee-7d45532fb1e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:06:25.543022 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e6837-a097-4100-b3ee-7d45532fb1e0 admin admin] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:06:25.543698 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e6837-a097-4100-b3ee-7d45532fb1e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:06:25.543912 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e6837-a097-4100-b3ee-7d45532fb1e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:06:25.544844 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e6837-a097-4100-b3ee-7d45532fb1e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:06:25.546461 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e6837-a097-4100-b3ee-7d45532fb1e0 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:06:25.556437 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1448/2881] 10.222.0.115 () {58 vars in 1127 bytes} [Tue Aug 2 11:06:25 2022] GET /identity/v3/roles => generated 1427 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 02 11:06:25.560783 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f5466-5ac6-4035-9359-14c7c8f650dd admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:06:25.571848 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f5466-5ac6-4035-9359-14c7c8f650dd admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:06:25.577221 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48110554-587f-421a-9bcd-829f4021e1ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:06:25.577897 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48110554-587f-421a-9bcd-829f4021e1ac admin admin] REQUEST_METHOD: `PUT` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:06:25.578177 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48110554-587f-421a-9bcd-829f4021e1ac admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:06:25.578439 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48110554-587f-421a-9bcd-829f4021e1ac admin admin] PATH_INFO: `/v3/projects/fe82126e018746fe8b88a098d205057d/users/1be384e468c24d5e93a53139fb697ec8/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:06:25.580509 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48110554-587f-421a-9bcd-829f4021e1ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe82126e018746fe8b88a098d205057d, user_id=1be384e468c24d5e93a53139fb697ec8, role_id=d81ad07a0a9e4f5f93dd6c3f2a29bcac)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:06:25.583345 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48110554-587f-421a-9bcd-829f4021e1ac admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:06:25.595875 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1434/2882] 10.222.0.115 () {60 vars in 1488 bytes} [Tue Aug 2 11:06:25 2022] PUT /identity/v3/projects/fe82126e018746fe8b88a098d205057d/users/1be384e468c24d5e93a53139fb697ec8/roles/d81ad07a0a9e4f5f93dd6c3f2a29bcac => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:06:31.525734 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4c06f-c11d-4a8d-90fc-f293e042ce28 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:06:31.525734 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4c06f-c11d-4a8d-90fc-f293e042ce28 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:06:31.526451 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf4c06f-c11d-4a8d-90fc-f293e042ce28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:06:31.561468 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-faf4c06f-c11d-4a8d-90fc-f293e042ce28 None None] MFA Rules not processed for user `1be384e468c24d5e93a53139fb697ec8`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:06:31.562826 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-faf4c06f-c11d-4a8d-90fc-f293e042ce28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:06:31.654908 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1449/2883] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:06:31 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:06:31.663382 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48110554-587f-421a-9bcd-829f4021e1ac admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:06:31.665098 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-48110554-587f-421a-9bcd-829f4021e1ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:06:31.751116 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48110554-587f-421a-9bcd-829f4021e1ac admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:06:31.754942 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64873807-6746-46f2-a845-09c7505bba87 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e358fd7c1b3c43d7ac89fb3562678266', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:06:31.755601 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64873807-6746-46f2-a845-09c7505bba87 service glance] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:06:31.755884 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64873807-6746-46f2-a845-09c7505bba87 service glance] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:06:31.756167 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64873807-6746-46f2-a845-09c7505bba87 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:06:31.834847 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64873807-6746-46f2-a845-09c7505bba87 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:06:31.836361 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64873807-6746-46f2-a845-09c7505bba87 service glance] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:06:31.854581 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1435/2884] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:06:31 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:06:32.130651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0aaef-4011-49fd-b296-f41b16c37672 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:06:32.130969 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0aaef-4011-49fd-b296-f41b16c37672 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:06:32.131414 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c0aaef-4011-49fd-b296-f41b16c37672 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:06:32.159522 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:06:32.185583 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-14c0aaef-4011-49fd-b296-f41b16c37672 None None] MFA Rules not processed for user `2b03263937cb4ca7807a24da44d37b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:06:32.195220 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-14c0aaef-4011-49fd-b296-f41b16c37672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:06:32.256686 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1450/2885] 10.222.0.115 () {58 vars in 1014 bytes} [Tue Aug 2 11:06:32 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:06:32.420658 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64873807-6746-46f2-a845-09c7505bba87 service glance] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:06:32.421822 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-64873807-6746-46f2-a845-09c7505bba87 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:06:32.425667 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:06:32.526160 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64873807-6746-46f2-a845-09c7505bba87 service glance] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:06:32.534173 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54539458-05de-465e-a7bb-f7955f6ecae8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0daa49b5a35f4a468bb8d0939e7ec20c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:06:32.539292 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54539458-05de-465e-a7bb-f7955f6ecae8 service swift] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:06:32.539292 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54539458-05de-465e-a7bb-f7955f6ecae8 service swift] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:06:32.539292 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54539458-05de-465e-a7bb-f7955f6ecae8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:06:32.544972 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54539458-05de-465e-a7bb-f7955f6ecae8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:06:32.546610 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54539458-05de-465e-a7bb-f7955f6ecae8 service swift] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:06:32.554739 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1436/2886] 10.222.0.115 () {60 vars in 1389 bytes} [Tue Aug 2 11:06:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 11:06:42.253481 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=32 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:06:42.444429 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834de18-8af9-4728-b59f-12a63c1b1dab None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:06:42.444738 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834de18-8af9-4728-b59f-12a63c1b1dab None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:06:42.445043 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5834de18-8af9-4728-b59f-12a63c1b1dab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:06:42.475232 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-5834de18-8af9-4728-b59f-12a63c1b1dab None None] MFA Rules not processed for user `7509ddedcec74149bb850e4adf64b489`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:06:42.476441 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-5834de18-8af9-4728-b59f-12a63c1b1dab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:06:42.483888 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1451/2887] 10.222.0.115 () {58 vars in 963 bytes} [Tue Aug 2 11:06:42 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:06:42.558795 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=40 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:06:44.144585 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54539458-05de-465e-a7bb-f7955f6ecae8 service swift] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:06:44.146022 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-54539458-05de-465e-a7bb-f7955f6ecae8 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:06:44.234824 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54539458-05de-465e-a7bb-f7955f6ecae8 service swift] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:06:44.240639 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a5f23-1494-4761-9a19-4574632f39b2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc4b962e2a742b08beef7187cc527e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:06:44.241461 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a5f23-1494-4761-9a19-4574632f39b2 service placement] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:06:44.241901 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a5f23-1494-4761-9a19-4574632f39b2 service placement] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:06:44.242315 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3a5f23-1494-4761-9a19-4574632f39b2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:06:44.244662 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-5f3a5f23-1494-4761-9a19-4574632f39b2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:06:44.260567 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3a5f23-1494-4761-9a19-4574632f39b2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:06:44.262623 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3a5f23-1494-4761-9a19-4574632f39b2 service placement] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:06:44.273448 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1437/2888] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:06:44 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:06:49.949105 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fec91b-01e5-48b9-b1dd-70d608be0ce0 None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:06:49.949644 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fec91b-01e5-48b9-b1dd-70d608be0ce0 None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:06:49.949644 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fec91b-01e5-48b9-b1dd-70d608be0ce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:06:49.970470 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-24fec91b-01e5-48b9-b1dd-70d608be0ce0 None None] MFA Rules not processed for user `2b03263937cb4ca7807a24da44d37b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:06:49.971940 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-24fec91b-01e5-48b9-b1dd-70d608be0ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:06:49.979394 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1452/2889] 10.222.0.115 () {58 vars in 1014 bytes} [Tue Aug 2 11:06:49 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:06:49.995574 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a5f23-1494-4761-9a19-4574632f39b2 service placement] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:06:50.007531 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3a5f23-1494-4761-9a19-4574632f39b2 service placement] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:06:50.013656 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2056cc2e-9117-42f0-b089-f3cab774f5da service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0daa49b5a35f4a468bb8d0939e7ec20c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:06:50.014556 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056cc2e-9117-42f0-b089-f3cab774f5da service swift] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:06:50.014849 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056cc2e-9117-42f0-b089-f3cab774f5da service swift] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:06:50.015162 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2056cc2e-9117-42f0-b089-f3cab774f5da service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:06:50.024316 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2056cc2e-9117-42f0-b089-f3cab774f5da service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:06:50.025914 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2056cc2e-9117-42f0-b089-f3cab774f5da service swift] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:06:50.035820 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1438/2890] 10.222.0.115 () {60 vars in 1389 bytes} [Tue Aug 2 11:06:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 11:06:59.964118 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=6 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:07:00.032414 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=38 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:07:03.616441 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fec91b-01e5-48b9-b1dd-70d608be0ce0 None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:07:03.618019 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-24fec91b-01e5-48b9-b1dd-70d608be0ce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:07:03.693821 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fec91b-01e5-48b9-b1dd-70d608be0ce0 None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:07:03.699506 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa462eb-abe1-4848-83d7-f07af94254df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:07:03.700165 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa462eb-abe1-4848-83d7-f07af94254df service nova] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:07:03.700480 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa462eb-abe1-4848-83d7-f07af94254df service nova] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:07:03.700777 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa462eb-abe1-4848-83d7-f07af94254df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:07:03.702532 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-1aa462eb-abe1-4848-83d7-f07af94254df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:07:03.715853 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa462eb-abe1-4848-83d7-f07af94254df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:07:03.717447 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa462eb-abe1-4848-83d7-f07af94254df service nova] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:07:03.729223 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1453/2891] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:07:03 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:07:13.724323 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=34 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:07:17.535944 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2056cc2e-9117-42f0-b089-f3cab774f5da service swift] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:07:17.537977 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-2056cc2e-9117-42f0-b089-f3cab774f5da service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:07:17.622398 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2056cc2e-9117-42f0-b089-f3cab774f5da service swift] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:07:17.627856 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca0e7b-8437-4827-9803-2fe5e7f06ad8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '98dcb74152ec4375a586ea526e7f07c8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:07:17.628267 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca0e7b-8437-4827-9803-2fe5e7f06ad8 service neutron] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:07:17.629121 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca0e7b-8437-4827-9803-2fe5e7f06ad8 service neutron] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:07:17.629121 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca0e7b-8437-4827-9803-2fe5e7f06ad8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:07:17.645278 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca0e7b-8437-4827-9803-2fe5e7f06ad8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:07:17.646723 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca0e7b-8437-4827-9803-2fe5e7f06ad8 service neutron] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:07:17.661808 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1439/2892] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:07:17 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:07:19.670142 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa462eb-abe1-4848-83d7-f07af94254df service nova] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:07:19.672356 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-1aa462eb-abe1-4848-83d7-f07af94254df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:07:19.688562 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa462eb-abe1-4848-83d7-f07af94254df service nova] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:07:19.694367 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a32d81f-4431-44e3-b113-ef9b694911cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd0478bd540f446ec810e2091c46ac49c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:07:19.694998 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32d81f-4431-44e3-b113-ef9b694911cc service nova] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:07:19.695403 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32d81f-4431-44e3-b113-ef9b694911cc service nova] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:07:19.695609 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a32d81f-4431-44e3-b113-ef9b694911cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:07:19.697380 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-0a32d81f-4431-44e3-b113-ef9b694911cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:07:19.710338 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a32d81f-4431-44e3-b113-ef9b694911cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:07:19.711976 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a32d81f-4431-44e3-b113-ef9b694911cc service nova] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:07:19.723599 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1454/2893] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:07:19 2022] GET /identity/v3/auth/tokens => generated 3382 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 02 11:07:27.656845 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=26 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:07:29.718951 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=6 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:08:06.008672 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca0e7b-8437-4827-9803-2fe5e7f06ad8 service neutron] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:08:06.010116 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-77ca0e7b-8437-4827-9803-2fe5e7f06ad8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:08:06.025378 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca0e7b-8437-4827-9803-2fe5e7f06ad8 service neutron] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:08:06.031000 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107348ca-0af5-4c98-8a33-9a0bed4e696e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc4b962e2a742b08beef7187cc527e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:08:06.031644 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107348ca-0af5-4c98-8a33-9a0bed4e696e service placement] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:08:06.031926 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107348ca-0af5-4c98-8a33-9a0bed4e696e service placement] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:08:06.032200 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107348ca-0af5-4c98-8a33-9a0bed4e696e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:08:06.033954 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-107348ca-0af5-4c98-8a33-9a0bed4e696e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:08:06.046566 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107348ca-0af5-4c98-8a33-9a0bed4e696e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:08:06.048171 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107348ca-0af5-4c98-8a33-9a0bed4e696e service placement] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:08:06.058267 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1440/2894] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:08:06 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:08:16.053940 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=6 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:08:32.759814 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1f1d2-7716-4c33-9118-dc53d51f0ffd None None] REQUEST_METHOD: `POST` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:08:32.760602 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1f1d2-7716-4c33-9118-dc53d51f0ffd None None] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:08:32.760602 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1f1d2-7716-4c33-9118-dc53d51f0ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:08:32.779934 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.auth.core [None req-74c1f1d2-7716-4c33-9118-dc53d51f0ffd None None] MFA Rules not processed for user `2b03263937cb4ca7807a24da44d37b0f`. Rule list: `[]` (Enabled: `True`). {{(pid=92111) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 02 11:08:32.781243 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-74c1f1d2-7716-4c33-9118-dc53d51f0ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:08:32.788466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1455/2895] 10.222.0.115 () {58 vars in 972 bytes} [Tue Aug 2 11:08:32 2022] POST /identity/v3/auth/tokens => generated 3205 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 02 11:08:32.831800 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107348ca-0af5-4c98-8a33-9a0bed4e696e service placement] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:08:32.843464 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107348ca-0af5-4c98-8a33-9a0bed4e696e service placement] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:08:32.848994 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272feb5-9974-40be-ac0b-564f4b426e92 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0daa49b5a35f4a468bb8d0939e7ec20c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:08:32.849607 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272feb5-9974-40be-ac0b-564f4b426e92 service swift] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:08:32.849890 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272feb5-9974-40be-ac0b-564f4b426e92 service swift] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:08:32.850165 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272feb5-9974-40be-ac0b-564f4b426e92 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:08:32.858439 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272feb5-9974-40be-ac0b-564f4b426e92 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:08:32.860677 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272feb5-9974-40be-ac0b-564f4b426e92 service swift] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:08:32.868777 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1441/2896] 10.222.0.115 () {60 vars in 1389 bytes} [Tue Aug 2 11:08:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 02 11:08:37.242251 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c1f1d2-7716-4c33-9118-dc53d51f0ffd None None] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:08:37.256427 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c1f1d2-7716-4c33-9118-dc53d51f0ffd None None] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:08:37.261232 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1380eb-c190-4777-8bae-5a26c3d27649 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc4b962e2a742b08beef7187cc527e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:08:37.261848 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1380eb-c190-4777-8bae-5a26c3d27649 service placement] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:08:37.262387 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1380eb-c190-4777-8bae-5a26c3d27649 service placement] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:08:37.262922 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1380eb-c190-4777-8bae-5a26c3d27649 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:08:37.264990 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-cc1380eb-c190-4777-8bae-5a26c3d27649 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:08:37.278907 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1380eb-c190-4777-8bae-5a26c3d27649 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:08:37.280651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1380eb-c190-4777-8bae-5a26c3d27649 service placement] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:08:37.304333 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1456/2897] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:08:37 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:08:40.903542 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272feb5-9974-40be-ac0b-564f4b426e92 service swift] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:08:40.915562 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272feb5-9974-40be-ac0b-564f4b426e92 service swift] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:08:40.920836 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a2570-a0a0-46c7-b928-714f86748061 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc4b962e2a742b08beef7187cc527e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:08:40.921488 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a2570-a0a0-46c7-b928-714f86748061 service placement] REQUEST_METHOD: `GET` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:08:40.922497 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a2570-a0a0-46c7-b928-714f86748061 service placement] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:08:40.922785 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a2570-a0a0-46c7-b928-714f86748061 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:08:40.924718 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-858a2570-a0a0-46c7-b928-714f86748061 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:08:40.937046 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a2570-a0a0-46c7-b928-714f86748061 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:08:40.938637 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a2570-a0a0-46c7-b928-714f86748061 service placement] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:08:40.949440 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1442/2898] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:08:40 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:08:41.041982 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1380eb-c190-4777-8bae-5a26c3d27649 service placement] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:08:41.109462 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1380eb-c190-4777-8bae-5a26c3d27649 service placement] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:08:41.113964 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e85bc-f0d6-4976-b900-e609ca645598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:08:41.114604 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e85bc-f0d6-4976-b900-e609ca645598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:08:41.114880 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e85bc-f0d6-4976-b900-e609ca645598 admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:08:41.115194 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e85bc-f0d6-4976-b900-e609ca645598 admin admin] PATH_INFO: `/v3/users/7509ddedcec74149bb850e4adf64b489` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:08:41.116353 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e85bc-f0d6-4976-b900-e609ca645598 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7509ddedcec74149bb850e4adf64b489)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:08:41.118317 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e85bc-f0d6-4976-b900-e609ca645598 admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:08:41.312466 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-fd3e85bc-f0d6-4976-b900-e609ca645598 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7509ddedcec74149bb850e4adf64b489'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:08:41.320762 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-fd3e85bc-f0d6-4976-b900-e609ca645598 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7509ddedcec74149bb850e4adf64b489'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:08:41.321535 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-fd3e85bc-f0d6-4976-b900-e609ca645598 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7509ddedcec74149bb850e4adf64b489'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:08:41.388953 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-fd3e85bc-f0d6-4976-b900-e609ca645598 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7509ddedcec74149bb850e4adf64b489'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:08:41.402651 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-fd3e85bc-f0d6-4976-b900-e609ca645598 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7509ddedcec74149bb850e4adf64b489'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:08:41.408999 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1457/2899] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:08:41 2022] DELETE /identity/v3/users/7509ddedcec74149bb850e4adf64b489 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:08:41.647082 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a2570-a0a0-46c7-b928-714f86748061 service placement] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:08:41.661346 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a2570-a0a0-46c7-b928-714f86748061 service placement] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:08:41.720913 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c105b9-44f7-4079-95ef-07415703f5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:08:41.721550 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c105b9-44f7-4079-95ef-07415703f5ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:08:41.721832 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c105b9-44f7-4079-95ef-07415703f5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:08:41.722107 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c105b9-44f7-4079-95ef-07415703f5ef admin admin] PATH_INFO: `/v3/projects/bb052a6c83084dd9bdbf408ad8870c16` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:08:41.723304 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c105b9-44f7-4079-95ef-07415703f5ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb052a6c83084dd9bdbf408ad8870c16)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:08:41.725210 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c105b9-44f7-4079-95ef-07415703f5ef admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:08:41.823204 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-29c105b9-44f7-4079-95ef-07415703f5ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb052a6c83084dd9bdbf408ad8870c16', 'request_id': 'req-29c105b9-44f7-4079-95ef-07415703f5ef'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:08:41.829730 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-29c105b9-44f7-4079-95ef-07415703f5ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb052a6c83084dd9bdbf408ad8870c16', 'request_id': 'req-29c105b9-44f7-4079-95ef-07415703f5ef'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:08:41.919965 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-29c105b9-44f7-4079-95ef-07415703f5ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb052a6c83084dd9bdbf408ad8870c16', 'request_id': 'req-29c105b9-44f7-4079-95ef-07415703f5ef'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:08:41.933694 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-29c105b9-44f7-4079-95ef-07415703f5ef admin admin] The token cache is being invalidate because project bb052a6c83084dd9bdbf408ad8870c16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:08:41.934168 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-29c105b9-44f7-4079-95ef-07415703f5ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:08:41.934863 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-29c105b9-44f7-4079-95ef-07415703f5ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:08:41.936436 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1443/2900] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:08:41 2022] DELETE /identity/v3/projects/bb052a6c83084dd9bdbf408ad8870c16 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:08:41.941572 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e85bc-f0d6-4976-b900-e609ca645598 admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:08:41.943067 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-fd3e85bc-f0d6-4976-b900-e609ca645598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:08:41.956735 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e85bc-f0d6-4976-b900-e609ca645598 admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:08:42.016562 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2c21c-a9fb-4882-905b-b01a7a2894ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:08:42.017591 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2c21c-a9fb-4882-905b-b01a7a2894ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:08:42.017591 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2c21c-a9fb-4882-905b-b01a7a2894ca admin admin] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:08:42.017817 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a2c21c-a9fb-4882-905b-b01a7a2894ca admin admin] PATH_INFO: `/v3/users/1be384e468c24d5e93a53139fb697ec8` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:08:42.018913 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a2c21c-a9fb-4882-905b-b01a7a2894ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=1be384e468c24d5e93a53139fb697ec8)` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:08:42.020928 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a2c21c-a9fb-4882-905b-b01a7a2894ca admin admin] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:08:42.022707 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:08:42.196509 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-88a2c21c-a9fb-4882-905b-b01a7a2894ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1be384e468c24d5e93a53139fb697ec8'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:08:42.204340 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-88a2c21c-a9fb-4882-905b-b01a7a2894ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1be384e468c24d5e93a53139fb697ec8'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:08:42.205058 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-88a2c21c-a9fb-4882-905b-b01a7a2894ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1be384e468c24d5e93a53139fb697ec8'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:08:42.268178 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-88a2c21c-a9fb-4882-905b-b01a7a2894ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1be384e468c24d5e93a53139fb697ec8'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:08:42.283412 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.notifications [None req-88a2c21c-a9fb-4882-905b-b01a7a2894ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1be384e468c24d5e93a53139fb697ec8'} {{(pid=92111) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:08:42.289641 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1458/2901] 10.222.0.115 () {58 vars in 1229 bytes} [Tue Aug 2 11:08:41 2022] DELETE /identity/v3/users/1be384e468c24d5e93a53139fb697ec8 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:08:42.556294 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c105b9-44f7-4079-95ef-07415703f5ef admin admin] Authenticating user token {{(pid=92110) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:08:42.556294 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.fernet_utils [None req-29c105b9-44f7-4079-95ef-07415703f5ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92110) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:08:42.569973 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c105b9-44f7-4079-95ef-07415703f5ef admin admin] Validating token access rules against request {{(pid=92110) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:08:42.633558 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854ca288-f792-43cb-be1d-994d42e4d464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4fb0c392df554f5dbd68f6861be954a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b86f954a6cac457bad0b226e93315c59', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92110) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:08:42.634222 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ca288-f792-43cb-be1d-994d42e4d464 admin admin] REQUEST_METHOD: `DELETE` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:08:42.634506 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ca288-f792-43cb-be1d-994d42e4d464 admin admin] SCRIPT_NAME: `/identity` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:08:42.635027 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854ca288-f792-43cb-be1d-994d42e4d464 admin admin] PATH_INFO: `/v3/projects/fe82126e018746fe8b88a098d205057d` {{(pid=92110) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:08:42.641358 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854ca288-f792-43cb-be1d-994d42e4d464 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe82126e018746fe8b88a098d205057d)` {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:08:42.641358 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854ca288-f792-43cb-be1d-994d42e4d464 admin admin] RBAC: Authorization granted {{(pid=92110) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:08:42.646282 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:08:42.763320 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-854ca288-f792-43cb-be1d-994d42e4d464 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe82126e018746fe8b88a098d205057d', 'request_id': 'req-854ca288-f792-43cb-be1d-994d42e4d464'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:08:42.770163 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-854ca288-f792-43cb-be1d-994d42e4d464 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe82126e018746fe8b88a098d205057d', 'request_id': 'req-854ca288-f792-43cb-be1d-994d42e4d464'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:08:42.867637 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-854ca288-f792-43cb-be1d-994d42e4d464 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe82126e018746fe8b88a098d205057d', 'request_id': 'req-854ca288-f792-43cb-be1d-994d42e4d464'} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:08:42.879765 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-854ca288-f792-43cb-be1d-994d42e4d464 admin admin] The token cache is being invalidate because project fe82126e018746fe8b88a098d205057d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=92110) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 02 11:08:42.882108 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-854ca288-f792-43cb-be1d-994d42e4d464 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:08:42.882108 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG keystone.notifications [None req-854ca288-f792-43cb-be1d-994d42e4d464 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=92110) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 02 11:08:42.882333 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: [pid: 92110|app: 0|req: 1444/2902] 10.222.0.115 () {58 vars in 1238 bytes} [Tue Aug 2 11:08:42 2022] DELETE /identity/v3/projects/fe82126e018746fe8b88a098d205057d => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 02 11:08:52.286046 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=26,keystone:DELETE=10,keystone:INSERT=1 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:08:53.062005 dv-cinder-850318-6-iscsi devstack@keystone.service[92110]: DEBUG dbcounter [-] [92110] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=92110) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 02 11:13:09.230157 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2c21c-a9fb-4882-905b-b01a7a2894ca admin admin] Authenticating user token {{(pid=92111) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 02 11:13:09.231614 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-88a2c21c-a9fb-4882-905b-b01a7a2894ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:13:09.369351 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a2c21c-a9fb-4882-905b-b01a7a2894ca admin admin] Validating token access rules against request {{(pid=92111) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 02 11:13:09.428221 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7bad0e-5f5b-4578-8191-b0e61dbbb0f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cbc4b962e2a742b08beef7187cc527e5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c1eae881e5714bf4bd223809e1a2d920', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=92111) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 02 11:13:09.428842 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bad0e-5f5b-4578-8191-b0e61dbbb0f0 service placement] REQUEST_METHOD: `GET` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 02 11:13:09.429123 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bad0e-5f5b-4578-8191-b0e61dbbb0f0 service placement] SCRIPT_NAME: `/identity` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 02 11:13:09.429428 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7bad0e-5f5b-4578-8191-b0e61dbbb0f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=92111) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 02 11:13:09.431260 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.fernet_utils [None req-fa7bad0e-5f5b-4578-8191-b0e61dbbb0f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=92111) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 02 11:13:09.443042 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7bad0e-5f5b-4578-8191-b0e61dbbb0f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 02 11:13:09.445796 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7bad0e-5f5b-4578-8191-b0e61dbbb0f0 service placement] RBAC: Authorization granted {{(pid=92111) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 02 11:13:09.456255 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: [pid: 92111|app: 0|req: 1459/2903] 10.222.0.115 () {60 vars in 1370 bytes} [Tue Aug 2 11:13:09 2022] GET /identity/v3/auth/tokens => generated 3318 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 02 11:13:19.450849 dv-cinder-850318-6-iscsi devstack@keystone.service[92111]: DEBUG dbcounter [-] [92111] Writing DB stats keystone:SELECT=89 {{(pid=92111) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}